Pyspark error passing StructType to Schema

Question

Getting Error : TypeError: 'StructType' object is not callable, while passing StructType to the schema method. Below is the code:

final_schema = StructType([StructField("id", StringType(), True)]) dataframe = sc.read.text('/path').schema(final_schema)

Data is string type as below:

id AO_01 AO_02 AO_03

whatsinthename · Accepted Answer · 2020-03-13 10:16:20Z

6

Changed the call to sc.read.schema(final_schema).text('/path') because text(...) will return a dataframe instance and need to set schema before that.

edited Mar 13, 2020 at 10:16

whatsinthename

2,1871 gold badge30 silver badges74 bronze badges

answered Mar 13, 2020 at 9:41

isha

1531 gold badge1 silver badge11 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Pyspark error passing StructType to Schema

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related