List of lists to rows in a pandas dataframe

Question

After using my script my algorithms return the exptected outcome in a list of lists like this: pred=[[b,c,d],[b,a,u],...[b,i,o]]

I already have a dataframe that needs those values added in a new matching column. The list is exactly x long like the other columns in the frame and I just need to create a new column with all the values of the lists.

However when I try to put the list into the column I get the error:

ValueError: Length of values does not match length of index

Looking at the data, it puts the entire list into one row instead of each entry in a new row.

EDIT:

All values in the list should be put in the column namend pred

sent token pred 0 a b 0 b c 0 b d 1 a b 1 b a 1 c u

Solution:

x = [] for _ in pred: if _ is not None: x += _ df_new = pd.DataFrame(df) df_new["pred"] = list(itertools.chain.from_iterable(x))

Can you show the code you have tried? As well as a sample input and expected output? — user3483203
– user3483203, Commented May 14, 2018 at 17:25

jpp · Accepted Answer · 2018-05-14 18:39:53Z

You can use itertools.chain, which allows you to flatten a list of lists, which you can then slice according to the length of your dataframe.

Data from @ak_slick.

import pandas as pd from itertools import chain df = pd.DataFrame({'sent': [0, 0, 0, 1, 1, 1], 'token': ['a', 'b', 'b', 'a', 'b', 'c']}) lst = [['b','c',None],['b',None,'u'], ['b','i','o']] df['pred'] = list(filter(None, chain.from_iterable(lst)))[:len(df.index)] print(df) sent token pred 0 0 a b 1 0 b c 2 0 b d 3 1 a b 4 1 b a 5 1 c u

Thank you! I had to clear NoneType entries first but this seems to work.
@ThelMi, I updated data for None values, just use filter(None, chain...). Note this method will remove all False values (e.g. None, 0, False, empty string, etc). Otherwise you can apply your own filtering logic.
I just applied a rule to my loop before. It seems to work and my classifiers seem to use the input perfectly. Thank you.

ak_slick · Accepted Answer · 2018-05-14 18:16:29Z

import pandas as pd # combine input lists x = [] for _ in [['b','c','d'],['b','a','u'], ['b','i','o']]: x += _ # output into a single column a = pd.Series(x) # mock original dataframe b = pd.DataFrame({'sent': [0, 0, 0, 1, 1, 1], 'token': ['a', 'b', 'b', 'a', 'b', 'c']}) # add column to existing dataframe # this will avoid the mis matched length error by ignoring anything longer # than your original data frame b['pred'] = a sent token pred 0 0 a b 1 0 b c 2 0 b d 3 1 a b 4 1 b a 5 1 c u

Collectives™ on Stack Overflow

List of lists to rows in a pandas dataframe

2 Answers 2

3 Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Related