Timeline for How to prefix columns names of dataframe efficiently without creating a new dataframe in Pyspark? [duplicate]
Current License: CC BY-SA 3.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Mar 23, 2021 at 13:47 | comment | added | Nidhi | This will do left_cols = df.columns 'df = df.selectExpr([col + ' as left_' + col for col in left_cols]) | |
| Jun 15, 2017 at 12:12 | vote | accept | GeorgeOfTheRF | ||
| Jun 15, 2017 at 11:36 | comment | added | zero323 | Aliasing creates a new DataFrame object, but it doesn't create a copy of the data. Unless you're worrying about local driver memory (in that case there is no good news for you) this is a duplicate. | |
| Jun 15, 2017 at 11:29 | history | closed | Yaron zero323 apache-spark Users with the apache-spark badge or a synonym can single-handedly close apache-spark questions as duplicates and reopen them as needed. | Duplicate of How to change dataframe column names in PySpark? | |
| Jun 15, 2017 at 10:22 | comment | added | GeorgeOfTheRF | No it doesnt because a new dataframe is created | |
| Jun 15, 2017 at 10:09 | answer | added | koiralo | timeline score: 1 | |
| Jun 15, 2017 at 9:41 | review | Close votes | |||
| Jun 15, 2017 at 11:34 | |||||
| Jun 15, 2017 at 9:24 | comment | added | Yaron | The answers in the linked question, seems to answer your question, e.g. data = data.select(col("Name").alias("name"), col("askdaosdka").alias("age")) | |
| Jun 15, 2017 at 9:23 | history | edited | GeorgeOfTheRF | CC BY-SA 3.0 | added 72 characters in body |
| Jun 15, 2017 at 9:23 | comment | added | GeorgeOfTheRF | Please read my question again. I have clearly mentioned how tha question is different from what I am asking. | |
| Jun 15, 2017 at 9:12 | history | asked | GeorgeOfTheRF | CC BY-SA 3.0 |