Skip to main content

Return to Question

added 5 characters in body; edited tags
Source Link
Oli
Oli
  • 10.5k
  • 5
  • 31
  • 51

Suppose I have a dataframe like this

val customer = Seq( ("C1", "Jackie Chan", 50, "Dayton", "M"), ("C2", "Harry Smith", 30, "Beavercreek", "M"), ("C3", "Ellen Smith", 28, "Beavercreek", "F"), ("C4", "John Chan", 26, "Dayton","M") ).toDF("cid","name","age","city","sex")

How can i get cid values in one column and get the rest of the values in an array < struct< column_name, column_value>>array < struct < column_name, column_value > > in spark

Suppose I have a dataframe like this

val customer = Seq( ("C1", "Jackie Chan", 50, "Dayton", "M"), ("C2", "Harry Smith", 30, "Beavercreek", "M"), ("C3", "Ellen Smith", 28, "Beavercreek", "F"), ("C4", "John Chan", 26, "Dayton","M") ).toDF("cid","name","age","city","sex")

How can i get cid values in one column and get the rest of the values in an array < struct< column_name, column_value>> in spark

Suppose I have a dataframe like this

val customer = Seq( ("C1", "Jackie Chan", 50, "Dayton", "M"), ("C2", "Harry Smith", 30, "Beavercreek", "M"), ("C3", "Ellen Smith", 28, "Beavercreek", "F"), ("C4", "John Chan", 26, "Dayton","M") ).toDF("cid","name","age","city","sex")

How can i get cid values in one column and get the rest of the values in an array < struct < column_name, column_value > > in spark

Source Link
Srinivas
Srinivas
  • 2.1k
  • 8
  • 31
  • 62

Converting a dataframe to an array of struct of column names and values

Suppose I have a dataframe like this

val customer = Seq( ("C1", "Jackie Chan", 50, "Dayton", "M"), ("C2", "Harry Smith", 30, "Beavercreek", "M"), ("C3", "Ellen Smith", 28, "Beavercreek", "F"), ("C4", "John Chan", 26, "Dayton","M") ).toDF("cid","name","age","city","sex")

How can i get cid values in one column and get the rest of the values in an array < struct< column_name, column_value>> in spark