df.to_dict('records') broken when 255+ columns present #24939
Description
Code Sample, a copy-pastable example if possible
Python 3.6.7 | packaged by conda-forge | (default, Nov 21 2018, 03:09:43) Type 'copyright', 'credits' or 'license' for more information IPython 7.2.0 -- An enhanced Interactive Python. Type '?' for help. In [1]: import pandas as pd In [2]: df = pd.DataFrame({('A_%d' % i): [i] for i in range(254)}) In [3]: r = df.to_dict('records') In [4]: len(r) Out[4]: 1 In [5]: len(r[0]) Out[5]: 254 In [6]: df_bad = pd.DataFrame({('A_%d' % i): [i] for i in range(255)}) In [7]: r_fail = df_bad.to_dict('records') --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) <ipython-input-7-4dd3de569735> in <module> ----> 1 r_fail = df_bad.to_dict('records') ~/.conda/envs/pandas-0.24/lib/python3.6/site-packages/pandas/core/frame.py in to_dict(self, orient, into) 1300 into_c((k, com.maybe_box_datetimelike(v)) 1301 for k, v in compat.iteritems(row._asdict())) -> 1302 for row in self.itertuples(index=False)] 1303 elif orient.lower().startswith('i'): 1304 if not self.index.is_unique: ~/.conda/envs/pandas-0.24/lib/python3.6/site-packages/pandas/core/frame.py in <listcomp>(.0) 1300 into_c((k, com.maybe_box_datetimelike(v)) 1301 for k, v in compat.iteritems(row._asdict())) -> 1302 for row in self.itertuples(index=False)] 1303 elif orient.lower().startswith('i'): 1304 if not self.index.is_unique: AttributeError: 'tuple' object has no attribute '_asdict' Problem description
Our test suite is failing with the latest release (0.24.0) on conda-forge. After looking into it, I've discovered that when 255 or more columns exist, the to_dict('records') method fails with AttributeError: 'tuple' object has no attribute '_asdict'.
Conda info
(pandas-0.24) 15:04:03 ~ § conda list # packages in environment at /home/evan/.conda/envs/pandas-0.24: # # Name Version Build Channel backcall 0.1.0 py_0 conda-forge blas 1.0 mkl ca-certificates 2018.11.29 ha4d7672_0 conda-forge certifi 2018.11.29 py36_1000 conda-forge decorator 4.3.0 py_0 conda-forge intel-openmp 2019.1 144 ipython 7.2.0 py36h24bf2e0_1000 conda-forge ipython_genutils 0.2.0 py_1 conda-forge jedi 0.13.2 py36_1000 conda-forge libffi 3.2.1 hf484d3e_1005 conda-forge libgcc-ng 7.3.0 hdf63c60_0 conda-forge libgfortran-ng 7.2.0 hdf63c60_3 conda-forge libstdcxx-ng 7.3.0 hdf63c60_0 conda-forge mkl 2019.1 144 mkl_fft 1.0.10 py36h14c3975_1 conda-forge mkl_random 1.0.2 py36h637b7d7_2 conda-forge ncurses 6.1 hf484d3e_1002 conda-forge numpy 1.15.4 py36h7e9f1db_0 numpy-base 1.15.4 py36hde5b4d6_0 openssl 1.0.2p h14c3975_1002 conda-forge pandas 0.24.0 py36hf484d3e_0 conda-forge parso 0.3.1 py_0 conda-forge pexpect 4.6.0 py36_1000 conda-forge pickleshare 0.7.5 py36_1000 conda-forge pip 18.1 py36_1000 conda-forge prompt_toolkit 2.0.7 py_0 conda-forge ptyprocess 0.6.0 py36_1000 conda-forge pygments 2.3.1 py_0 conda-forge python 3.6.7 hd21baee_1001 conda-forge python-dateutil 2.7.5 py_0 conda-forge pytz 2018.9 py_0 conda-forge readline 7.0 hf8c457e_1001 conda-forge setuptools 40.6.3 py36_0 conda-forge six 1.12.0 py36_1000 conda-forge sqlite 3.26.0 h67949de_1000 conda-forge tk 8.6.9 h84994c4_1000 conda-forge traitlets 4.3.2 py36_1000 conda-forge wcwidth 0.1.7 py_1 conda-forge wheel 0.32.3 py36_0 conda-forge xz 5.2.4 h14c3975_1001 conda-forge zlib 1.2.11 h14c3975_1004 conda-forge (pandas-0.24) 15:04:04 ~ § conda info active environment : pandas-0.24 active env location : /home/evan/.conda/envs/pandas-0.24 shell level : 1 user config file : /home/evan/.condarc populated config files : conda version : 4.5.12 conda-build version : 3.17.6 python version : 3.7.2.final.0 base environment : /opt/Miniconda3 (read only) channel URLs : https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/free/linux-64 https://repo.anaconda.com/pkgs/free/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch https://repo.anaconda.com/pkgs/pro/linux-64 https://repo.anaconda.com/pkgs/pro/noarch package cache : /opt/Miniconda3/pkgs /home/evan/.conda/pkgs envs directories : /home/evan/.conda/envs /opt/Miniconda3/envs platform : linux-64 user-agent : conda/4.5.12 requests/2.19.1 CPython/3.7.2 Linux/4.20.3-arch1-1-ARCH antergos/ glibc/2.28 UID:GID : 1000:985 netrc file : None offline mode : False Output of pd.show_versions()
INSTALLED VERSIONS ------------------ commit: None python: 3.6.7.final.0 python-bits: 64 OS: Linux OS-release: 4.20.3-arch1-1-ARCH machine: x86_64 processor: byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8 pandas: 0.24.0 pytest: None pip: 18.1 setuptools: 40.6.3 Cython: None numpy: 1.15.4 scipy: None pyarrow: None xarray: None IPython: 7.2.0 sphinx: None patsy: None dateutil: 2.7.5 pytz: 2018.9 blosc: None bottleneck: None tables: None numexpr: None feather: None matplotlib: None openpyxl: None xlrd: None xlwt: None xlsxwriter: None lxml.etree: None bs4: None html5lib: None sqlalchemy: None pymysql: None psycopg2: None jinja2: None s3fs: None fastparquet: None pandas_gbq: None pandas_datareader: None gcsfs: None