TST: Difference in drop_duplicates for numeric Series and DataFrames #14192
Description
Code Sample
import pandas as pd import numpy as np df = pd.DataFrame({'a':[1,1,1,'one', 'one'], 'b': [2,2,np.nan, np.nan, np.nan], 'c': [3,3,np.nan, np.nan, 'three'] }) print df[['b']].drop_duplicates() print df['b'].drop_duplicates() Expected Output
Why the result is not consistent?
For a dataframe (first case), the NA values are considered equal (we get 2 unique values), while for a series (second case), the NA values are not (similarly to numpy.unique, we get 3 unique values)
Curiously, in the case of a mixed type column, len(df[['c']].drop_duplicates() ) == len(df['c'].drop_duplicates())
output of pd.show_versions()
INSTALLED VERSIONS ------------------ commit: None python: 2.7.12.final.0 python-bits: 64 OS: Darwin OS-release: 15.6.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: en_US.UTF-8 LANG: en_US.UTF-8 pandas: 0.18.1 nose: None pip: 8.0.3 setuptools: 20.1.1 Cython: 0.23.4 numpy: 1.11.0 scipy: 0.17.1 statsmodels: None xarray: None IPython: 4.1.2 sphinx: None patsy: None dateutil: 2.5.3 pytz: 2016.4 blosc: None bottleneck: None tables: None numexpr: None matplotlib: 1.5.1 openpyxl: None xlrd: None xlwt: None xlsxwriter: None lxml: None bs4: None html5lib: None httplib2: 0.9.2 apiclient: None sqlalchemy: None pymysql: None psycopg2: None jinja2: 2.8 boto: None pandas_datareader: None