Skip to content

read_csv: if parse_dates dont appear in use_cols, we get a trace #31251

@teto

Description

@teto

Code Sample, a copy-pastable example if possible

# Your code here import pandas as pd import io content = io.StringIO(''' time,val 212.23, 32 ''') date_cols = ['time'] df = pd.read_csv( content, sep=',', usecols=['val'], dtype= { 'val': int }, parse_dates=date_cols, )

Problem description

triggers

Traceback (most recent call last): File "test.py", line 16, in <module> parse_dates=date_cols, File "/nix/store/k4fd48jzsyafvcifa6wi6pk4vaprnw36-python3.7-pandas-0.25.3/lib/python3.7/site-packages/pandas/io/parsers.py", line 685, in parser_f return _read(filepath_or_buffer, kwds) File "/nix/store/k4fd48jzsyafvcifa6wi6pk4vaprnw36-python3.7-pandas-0.25.3/lib/python3.7/site-packages/pandas/io/parsers.py", line 463, in _read data = parser.read(nrows) File "/nix/store/k4fd48jzsyafvcifa6wi6pk4vaprnw36-python3.7-pandas-0.25.3/lib/python3.7/site-packages/pandas/io/parsers.py", line 1154, in read ret = self._engine.read(nrows) File "/nix/store/k4fd48jzsyafvcifa6wi6pk4vaprnw36-python3.7-pandas-0.25.3/lib/python3.7/site-packages/pandas/io/parsers.py", line 2134, in read names, data = self._do_date_conversions(names, data) File "/nix/store/k4fd48jzsyafvcifa6wi6pk4vaprnw36-python3.7-pandas-0.25.3/lib/python3.7/site-packages/pandas/io/parsers.py", line 1885, in _do_date_conversions keep_date_col=self.keep_date_col, File "/nix/store/k4fd48jzsyafvcifa6wi6pk4vaprnw36-python3.7-pandas-0.25.3/lib/python3.7/site-packages/pandas/io/parsers.py", line 3335, in _process_date_conversion data_dict[colspec] = converter(data_dict[colspec]) KeyError: 'time' 

i.e., if you use columns in parse_dates that dont appear in use_cols, then you are screwed.

Either parse_dates sould be added to use_cols or the documentation precise it. An assert could make the error more understandable too.
pandas 0.25.3

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions