How can I compare rows for multiple columns if they are identical

Question

I have one pandas DataFrame and I want to check the rows are identical. This shall work for 3 columns out of the dataframe.

A B C ... 1 1 1 ... 1 2 3 ... 1 4 4 ...

This should bring up a mask with

True False False

MaxU - stand with Ukraine · Accepted Answer · 2017-11-17 20:54:44Z

5

In [97]: df.eq(df.iloc[:, 0], axis=0).all(1) Out[97]: 0 True 1 False 2 False dtype: bool

answered Nov 17, 2017 at 20:54

212k37 gold badges402 silver badges436 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

BENY · Accepted Answer · 2017-11-17 20:57:49Z

4

Or you can use nunique

df.nunique(1).eq(1) Out[1155]: 0 True 1 False 2 False dtype: bool

answered Nov 17, 2017 at 20:57

324k22 gold badges176 silver badges250 bronze badges

Very nice! I didn't know we have such function. (It was added in Pandas 0.20.0)

@MaxU to be honestly , I do not think those function are needed..., Your method is faster, I tested it before :-)

Yes... I really like this one. And I have a derivative solution. At least inspired.

piRSquared · Accepted Answer · 2017-11-17 21:22:40Z

Using diff

df.diff(axis=1).fillna(0).eq(0).all(1) 0 True 1 False 2 False dtype: bool

Using shift

df.eq(df.shift(axis=1)).iloc[:, 1:].all(1) 0 True 1 False 2 False dtype: bool

Using std (inspired by @Wen)

df.std(1).eq(0) 0 True 1 False 2 False dtype: bool

Inspired by @MaxU with a healthy amount of obnoxious sprinkled in.

(lambda v, j: pd.Series((v == v[:, [0]]).all(1), j))(df.values, df.index) 0 True 1 False 2 False dtype: bool

Now with less obnoxious

v = df.values pd.Series((v == v[:, [0]]).all(1), df.index)