pandas: combine two columns in a DataFrame

you can use directly fillna and assigning the result to the column 'bar'

df['bar'].fillna(df['foo'], inplace=True) del df['foo'] 

general example:

import pandas as pd #creating the table with two missing values df1 = pd.DataFrame({'a':[1,2],'b':[3,4]}, index = [1,2]) df2 = pd.DataFrame({'b':[5,6]}, index = [3,4]) dftot = pd.concat((df1, df2)) print dftot #creating the dataframe to fill the missing values filldf = pd.DataFrame({'a':[7,7,7,7]}) #filling print dftot.fillna(filldf) 

More modern pandas versions (since at least 0.12) have the combine_first() and update() methods for DataFrame and Series objects. For example if your DataFrame were called df, you would do:

df.bar.combine_first(df.foo) 

which would only alter Nan values of the bar column to match the foo column, and would do so inplace. To overwrite non-Nan values in bar with those in foo, you would use the update() method.

Another option, use the .apply() method on the frame. You can do reassign a column with deference to existing data...

import pandas as pd import numpy as np # get your data into a dataframe # replace content in "bar" with "foo" if "bar" is null df["bar"] = df.apply(lambda row: row["foo"] if row["bar"] == np.NaN else row["bar"], axis=1) # note: change 'np.NaN' with null values you have like an empty string 

You can do this using numpy too.

df['bar'] = np.where(pd.isnull(df['bar']),df['foo'],df['bar'])