I have an column containing an array of phone numbers represented as structs, and need to put them into three columns by a "type" attribute (phone1, phone2, fax).
Here are two sample values of the column.
[{"number":"000-000-0000","type":"Phone1"},{"number":"000-000-0001","type":"Phone2"},{"number":"000-000-0002","type":"Fax"}] [{"number":"000-000-1000","type":"Phone1"},{"number":"000-000-1001","typeCode":"Fax"},{"number":"000-000-1002","type":"Phone2"}] I want to split each into three columns, one for each type. I want something like this:
Phone1 Phone2 Fax 000-000-0000 000-000-0001 000-000-0002 000-000-1000 000-000-1002 000-000-1001 This answer shows how to put each element of the array into its own column. How to explode an array into multiple columns in Spark
This gets me halfway there, but I can't rely on the order of items in the array. If I do this I'll get something like this, where the Phone2 and Fax values in the second column are out of place.
Phone1 Phone2 Fax 000-000-0000 000-000-0001 000-000-0002 000-000-1000 000-000-1001 000-000-1002 How can I split the single column value into three columns, using the type value? An array can have 0-3 numbers, but will never have more than one number of each type.
