More specific Pandas testing for equality between two dataframes

Question

I want to know how the value looks in df1, and how the value looks in df2, when the two dfs fail equality tests.

This is the most specific output I could get:

# more specific cell by cell test Assertion_df = Output_Test_Cases_df.merge(Correct_Test_Cases_df).eq(Output_Test_Cases_df) print(Assertion_df.iloc[:, [1, 2, 3]])

 DBN AI Code Active AI 0 True True True 1 False True True 2 True True True 3 True True True 4 True False True 5 True True True 6 True True True 7 True True True

However, I would like something like

 DBN AI Code Active AI 0 True True True 1 12X000 in df1, 12X111 in df 2 True True 2 True True True 3 True True True 4 True 100 in df1, 200 in df2 True 5 True True True 6 True True True 7 True True True

Anything functionally equivalent to the above would be appreciated.

Provide your input data as well.

Erfan
– Erfan

2019-07-26 17:05:51 +00:00
Commented Jul 26, 2019 at 17:05 — Erfan
– Erfan, Commented Jul 26, 2019 at 17:05

manesioz · Accepted Answer · 2019-07-26 17:05:53Z

You could try something like

dfBool = (Correct_Test_Cases_df != Output_Test_Cases_df).stack() # Create Frame of comparison booleans dfDiff = pd.concat([Correct_Test_Cases_df.stack()[dfBool], Output_Test_Cases_df.stack()[dfBool]], axis=1) dfDiff.columns=["Old", "New"] print(dfDiff)

ALollz · Accepted Answer · 2019-07-26 18:05:00Z

You can write your own function, after using mask to select only the unequal cells. I'd use this just for visualization, it's far easier to deal with individual values with just df1.mask(s)

Code:

def my_agg(x): if x.isnull().all(): return True else: return list(zip(x, x.index.get_level_values(0))) s = df1 == df2 (pd.concat([df1.mask(s), df2.mask(s)], keys=['df1', 'df2']) .groupby(level=1).agg(my_agg)) # A B C #0 True True True #1 True True True #2 [(c, df1), (b, df2)] True [(3.0, df1), (1.0, df2)] #3 True True True #4 True [(E, df1), (X, df2)] True

Sample Data:

import pandas as pd df1 = pd.DataFrame({'A': list('abcde'), 'B': list('ABCDE'), 'C': [1,2,3,4,5]}) df2 = pd.DataFrame({'A': list('abbde'), 'B': list('ABCDX'), 'C': [1,2,1,4,5]})

Collectives™ on Stack Overflow

More specific Pandas testing for equality between two dataframes

2 Answers 2

Comments

Code:

Sample Data:

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Code:

Sample Data:

Comments

Related