How to use pd.apply() to instantiate new columns?

Question

Instead of doing this:

df['A'] = df['A'] if 'A' in df else None df['B'] = df['B'] if 'B' in df else None df['C'] = df['C'] if 'C' in df else None df['D'] = df['D'] if 'D' in df else None ...

I want to do this in one line or function. Below is what I tried:

def populate_columns(df): col_names = ['A', 'B', 'C', 'D', 'E', 'F', ...] def populate_column(df, col_name): df[col_name] = df[col_name] if col_name in df else None return df[col_name] df[col_name] = df.apply(lambda x: populate_column(x) for x in col_names) return df

But I just get Exception has occurred: ValueError. What can I do here?

mozway · Accepted Answer · 2022-12-02 05:12:31Z

1

Looks like you can replace your whole code with a reindex:

ensure_cols = ['A', 'B', 'C', 'D'] df = df.reindex(columns=df.columns.union(ensure_cols))

NB. By default the fill value is NaN, if you really want None use fill_value=None.

If you want to fix your code, just use a single loop:

col_names = ['A', 'B', 'C', 'D'] for c in col_names: if c not in df: df[c] = None

answered Dec 2, 2022 at 5:12

mozway

267k13 gold badges56 silver badges106 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

amnesic Over a year ago

I tried using fill_value=None but still returns NaN

mozway Over a year ago

@amnesic strange, indeed. Then a workaround could be: reindex(..., fill_value='None').replace('None', None) or use any string that you know is not in the data if 'None' can be.

amnesic Over a year ago

What's the point of self.df.columns.union(ensure_columns)? I can just pass in ensure columns, can't I?

mozway Over a year ago

It is just in case you have original columns that are not in the list.

Collectives™ on Stack Overflow

How to use pd.apply() to instantiate new columns?

1 Answer 1

4 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Related