41

I am using CassandraSQLContext from spark-shell to query data from Cassandra. So, I want to know two things one how to fetch more than 20 rows using CassandraSQLContext and second how do Id display the full value of column. As you can see below by default it append dots in the string values.

Code :

val csc = new CassandraSQLContext(sc) csc.setKeyspace("KeySpace") val maxDF = csc.sql("SQL_QUERY" ) maxDF.show

Output:

+--------------------+--------------------+-----------------+--------------------+ | id| Col2| Col3| Col4| +--------------------+--------------------+-----------------+--------------------+ |8wzloRMrGpf8Q3bbk...| Value1| X| K1| |AxRfoHDjV1Fk18OqS...| Value2| Y| K2| |FpMVRlaHsEOcHyDgy...| Value3| Z| K3| |HERt8eFLRtKkiZndy...| Value4| U| K4| |nWOcbbbm8ZOjUSNfY...| Value5| V| K5|
Improve this question
0

3 Answers 3

Reset to default
71

If you want to print the whole value of a column, in scala, you just need to set the argument truncate from the show method to false :

maxDf.show(false)

and if you wish to show more than 20 rows : 

// example showing 30 columns of // maxDf untruncated maxDf.show(30, false)

For pyspark, you'll need to specify the argument name :

maxDF.show(truncate = False)
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

For pyspark users who make it here, just be sure to specify the truncate argument name and capitalize your False: maxDF.show(truncate = False)
6

You won't get in nice tabular form instead it will be converted to scala object.

maxDF.take(50)
Improve this answer

Comments

3

To show all rows of a dataframe, following can help :

df.show(n=df.count(), truncate=False)

Please be careful not to use this on a big dataFrame.

Quick reference on the parameters to show method (pyspark version):

Parameters ---------- n : int, optional Number of rows to show. truncate : bool or int, optional If set to ``True``, truncate strings longer than 20 chars by default. If set to a number greater than one, truncates long strings to length ``truncate`` and align cells right.
Improve this answer

1 Comment

The only sensible answer, thanks!

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.