Skip to content

Conversation

@tapiralec
Copy link

Added support for millis resolution on a parsed DatetimeIndex slice.

There was no elif statement for when _parsed_string_to_bounds(self,
reso, parsed) was given reso == 'millisecond'

  • passes git diff upstream/master -u -- "*.py" | flake8 --diff
Added support for millis resolution on a parsed DatetimeIndex slice. There was no elif statement for when _parsed_string_to_bounds(self, reso, parsed) was given reso == 'millisecond'
@pep8speaks
Copy link

Hello @tapiralec! Thanks for submitting the PR.

@codecov
Copy link

codecov bot commented Oct 5, 2018

Codecov Report

Merging #23010 into master will decrease coverage by <.01%.
The diff coverage is 33.33%.

Impacted file tree graph

@@ Coverage Diff @@ ## master #23010 +/- ## ========================================== - Coverage 92.19% 92.19% -0.01%  ========================================== Files 169 169 Lines 50835 50838 +3 ========================================== + Hits 46868 46869 +1  - Misses 3967 3969 +2
Flag Coverage Δ
#multiple 90.61% <33.33%> (-0.01%) ⬇️
#single 42.35% <0%> (-0.01%) ⬇️
Impacted Files Coverage Δ
pandas/core/indexes/datetimes.py 95.53% <33.33%> (-0.25%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d523d9f...8109682. Read the comment docs.

@jreback
Copy link
Contributor

jreback commented Oct 6, 2018

is this fixing an issue? this would need tests, I am not sure what doesn't work now.

@gfyoung gfyoung added Datetime Datetime data dtype Indexing Related to indexing on series/frames, not to indexes themselves DataFrame DataFrame data structure Bug labels Oct 7, 2018
@tapiralec
Copy link
Author

Sorry for the lack of good explanation earlier. I couldn't find any existing bug in the issues that listed this problem. Basically there exists support for slicing data with a DatetimeIndex using strings for the range in core/indexes/datetimes.py (line 1263):

_, parsed, reso = parsing.parse_time_string(label, freq) lower, upper = self._parsed_string_to_bounds(reso, parsed) 

_libs/tslibs/parsing.pyx returns a resolution of milliseconds when microsecond % 1000 == 0 (line 334)
_parsed_string_to_bounds() has if/elif's on the resolution argument for year, month, ... second, microsecond, but doesn't have one for millisecond.

I found this when I was working with data that had a precision of tenths of milliseconds (hundreds of microseconds). If you pass in a string bound with, e.g. .9871 for the fraction of the second, it slices as expected since 987100 % 1000 = 100 != 0, so resolution is "microseconds". But if you pass in a string bound with .9870, the parsed resolution is determined to be milliseconds and _parsed_string_to_bounds() will raise a KeyError.

With regards to the tests: I'm not familiar yet with pytest / numpy.testing, but I can take a crack at writing the tests to cover this case.

Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs tests and a whatsnew

@jreback
Copy link
Contributor

jreback commented Nov 23, 2018

closing as stale. if you'd like to continue, pls ping.

@jreback jreback closed this Nov 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug DataFrame DataFrame data structure Datetime Datetime data dtype Indexing Related to indexing on series/frames, not to indexes themselves

4 participants