rename columns in dataframe pyspark adding a string

Question

I have written code in Python using Pandas that adds "VEN_" to the beginning of the column names:

Tablon.columns = "VEN_" + Tablon.columns

And it works fine, but now I'm working with PySpark and it doesn't work. I've tried:

Vaa_total.columns = ['Vaa_' + col for col in Vaa_total.columns]

or

for elemento in Vaa_total.columns: elemento = "Vaa_" + elemento

And other things like that but it doesn't work.

I don't want to replace the columns name, I just want to mantain it but adding a string to the beginning.

Possible duplicate of How to change dataframe column names in pyspark? — vvg
– vvg, Commented Jul 17, 2018 at 8:40
I don't think so, there is explained how to replace it but I don't know how I can to add a string to my columns name, I get: AttributeError: can't set attribute. — ITo
– ITo, Commented Jul 17, 2018 at 8:46

ags29 · Accepted Answer · 2018-07-17 08:50:31Z

Try something like this:

for elemento in Vaa_total.columns: Vaa_total =Vaa_total.withColumnRenamed(elemento, "Vaa_" + elemento)

vvg · Accepted Answer · 2018-07-17 08:52:51Z

I linked similar topic in comment. Here's example adapted from that topic to your task:

dataframe.select([col(col_name).alias('VAA_' + col_name) for col_name in dataframe])

sargupta · Accepted Answer · 2022-04-02 10:00:48Z

0

Standard format of writing it:

renamed_df = df.withColumnRenamed(col_name, "insert_text" + col_name) for col_name in dataframe.columns])

answered Apr 2, 2022 at 10:00

1,03316 silver badges28 bronze badges

1 Comment

There is a bracket missing somewhere in your solution