pandas concat to merge list of dataframes to another dataframe

Question

I have dataframes named a, b, c (and more), and I'd like to concat them with a datafram named d.

I'm doing as below:

a= pd.concat([a, d], axis = 0) b= pd.concat([b, d], axis = 0) c= pd.concat([c, d], axis = 0) ...

As I have more tham 3 dataframe a,b,c, I'm looking for some ways to do it faster, like in a loop for eg. I'm using code as below but it doesn't work:

for x in [a,b,c]: global d x = pd.concat([x, d], axis = 0)

You need to write your own merge function, where you check weather d is empty. If so return x, else: concat x and d as shown. — Tom S
– Tom S, Commented Aug 13, 2021 at 11:01
Instead of having your three dataframe stored in unique variables, could you instead store them to a list or a dictionary? — Scott Boston
– Scott Boston, Commented Aug 13, 2021 at 13:09

BlackMath · Accepted Answer · 2021-08-13 12:45:28Z

import pandas as pd a = pd.DataFrame({'task':['one'],'label':['tag_01'] }) b = pd.DataFrame({'task':['one'],'label':['tag_01'] }) c = pd.DataFrame({'task':['one'],'label':['tag_01'] }) d = pd.DataFrame({'task':['two'],'label':['tag_02'] })

A common solution to iterate over dataframes while using their name is to use another list with dataframes names, then enumerate and get value of name by position:

# create a dict containing dfs names = ['a','b','c'] dict_dfs = ({names[i]: df for i, df in enumerate([a, b, c])})

Now we can use globals() to generate desidered output. The globals() method returns the dictionary of the current global symbol table. A symbol table is a data structure maintained by a compiler.

for i in dict_dfs: globals()[str(i)] = pd.concat([dict_dfs[i], d], axis = 0)

Results:

a Out[1]: task label 0 one tag_01 0 two tag_02 b Out[2]: task label 0 one tag_01 0 two tag_02 c Out[3]: task label 0 one tag_01 0 two tag_02

Collectives™ on Stack Overflow

pandas concat to merge list of dataframes to another dataframe

1 Answer 1

3 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Related