BUG: DataFrame.asof() : Timezone Awareness / Naivety comparison TypeError (incorrect) #21284

msmarchena · 2018-06-01T11:32:43Z

closes DataFrame.asof() : Timezone Awareness / Naivety comparison TypeError (incorrect) #21194
tests added / passed
whatsnew entry
- modified whatsnew.v0.23.1.txt

pep8speaks · 2018-06-01T11:32:46Z

Hello @msmarchena! Thanks for updating the PR.

In the file pandas/io/tests/test_excel.py, following are the PEP8 issues :

Line 2193:13: E722 do not use bare except'

Comment last updated on August 03, 2018 at 14:02 Hours UTC

jschendel · 2018-06-01T16:42:32Z

doc/source/whatsnew/v0.23.1.txt

 Bug Fixes
 ~~~~~~~~~

+- Bug in :func:`Dataframe.asof` that reaised ``TypeError`` when attempting to compare tz-naive and tz-aware timestamps (:issue:`21194`)


"reaised" --> "raised a"

codecov · 2018-06-01T16:44:57Z

Codecov Report

Merging #21284 into master will not change coverage.
The diff coverage is 100%.

@@ Coverage Diff @@ ## master #21284 +/- ## ======================================= Coverage 91.85% 91.85% ======================================= Files 153 153 Lines 49546 49546 ======================================= Hits 45509 45509 Misses 4037 4037

Flag	Coverage Δ
#multiple	`90.25% <100%> (ø)`	⬆️
#single	`41.87% <0%> (ø)`	⬆️

Impacted Files	Coverage Δ
pandas/core/indexes/base.py	`96.61% <100%> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4274b84...8441881. Read the comment docs.

mroeschke · 2018-06-01T20:52:08Z

pandas/tests/frame/test_asof.py

+ # Testing if DataFrame index preserve timezone (UTC)
+ df = DataFrame(data=[1, 2], index=[timestamp1, timestamp2])
+ result = df.asof(timestamp_internal)
+ expected = Series(2.0, index=RangeIndex(start=0, stop=1, step=1),


RangeIndex isn't explicitly needed to construct the Series I think. It's created by default.

mroeschke · 2018-06-01T20:54:36Z

pandas/tests/frame/test_asof.py

 expected = Series(np.nan, index=['A', 'B'], name=3)
 tm.assert_series_equal(result, expected)
+
+ def test_time_zone_aware_index(self):


If possible, could you use pytest.mark.parameterize to combine these two tests? The structure of these tests seem very similar. Looks like you might be able to param over timestamp_internal

jreback · 2018-06-04T21:36:09Z

doc/source/whatsnew/v0.23.1.txt

 Bug Fixes
 ~~~~~~~~~

+- Bug in :func:`Dataframe.asof` that raised a ``TypeError`` when attempting to compare tz-naive and tz-aware timestamps (:issue:`21194`)


move under the Conversion section

jreback · 2018-06-04T21:37:10Z

pandas/tests/frame/test_asof.py

+ # Testing awareness of DataFrame index considering different
+ # UTC and different timezone
+ @pytest.mark.parametrize("stamp,expected",
+ [(Timestamp('2018-01-01 23:22:43.325+00:00'),


this will have some linting errors

you can format this a bit better by doing

@pytest.mark.parameterize( "stamp, expected", [ (......), (......)])

jreback · 2018-06-04T21:38:08Z

pandas/tests/frame/test_asof.py

+ @pytest.mark.parametrize("stamp,expected",
+ [(Timestamp('2018-01-01 23:22:43.325+00:00'),
+ Series(2.0,name=Timestamp('2018-01-01 23:22:43.325+00:00'))),
+ (Timestamp('2018-01-01 22:33:20.682+01:00'),


can you add a NaT in here as well (so another case)

I don't understand what do you mean. Where exactly I should add a NaT?

equiv of the naive case, but with a non-naive Timestamp (we may already have this test, if so, just add on to it)

In [2]: df = DataFrame(data=[1, 2], ...: index=[pd.Timestamp('2018-01-01 21:00:05.001'), ...: pd.Timestamp('2018-01-01 22:35:10.550')]) ...: In [3]: df Out[3]: 0 2018-01-01 21:00:05.001 1 2018-01-01 22:35:10.550 2 In [4]: df.asof('2018-01-01 22:35:10.550') Out[4]: 0 2.0 Name: 2018-01-01 22:35:10.550000, dtype: float64 In [5]: df.asof(pd.NaT) Out[5]: 0 1.0 Name: NaT, dtype: float64 In [7]: df.asof('2018-01-01') Out[7]: 0 NaN Name: 2018-01-01 00:00:00, dtype: float64

thanks for the clarification. I will check it out.

with the fix where.values in asof_loc I have for the NaT case the following:

df Out[18]: 0 2018-01-01 21:00:05.001 1 2018-01-01 22:35:10.550 2 result=df.asof(pd.NaT) /Users/.../pandas/core/indexes/base.py:2382: FutureWarning: In the future, 'NAT < x' and 'x < NAT' will always be False. result[(locs == 0) & (where.values < self.values[first])] = -1 result Out[20]: 0 NaN Name: NaT, dtype: float64

Is 1 really the desired value in this situation?

jreback · 2018-06-05T11:06:03Z

pandas/tests/frame/test_asof.py

+ @pytest.mark.parametrize("stamp,expected",
+ [(Timestamp('2018-01-01 23:22:43.325+00:00'),
+ Series(2.0,name=Timestamp('2018-01-01 23:22:43.325+00:00'))),
+ (Timestamp('2018-01-01 22:33:20.682+01:00'),


equiv of the naive case, but with a non-naive Timestamp (we may already have this test, if so, just add on to it)

In [2]: df = DataFrame(data=[1, 2], ...: index=[pd.Timestamp('2018-01-01 21:00:05.001'), ...: pd.Timestamp('2018-01-01 22:35:10.550')]) ...: In [3]: df Out[3]: 0 2018-01-01 21:00:05.001 1 2018-01-01 22:35:10.550 2 In [4]: df.asof('2018-01-01 22:35:10.550') Out[4]: 0 2.0 Name: 2018-01-01 22:35:10.550000, dtype: float64 In [5]: df.asof(pd.NaT) Out[5]: 0 1.0 Name: NaT, dtype: float64 In [7]: df.asof('2018-01-01') Out[7]: 0 NaN Name: 2018-01-01 00:00:00, dtype: float64

jreback · 2018-06-19T00:48:55Z

can you rebase and update

…pr/12051

…R21194

jreback · 2018-08-01T01:11:08Z

can u rebase

grahamjeffries and others added 8 commits September 30, 2016 11:13

remove read_excel kwd NotImplemented error, update documentation #11544

0e10a9d

Update tests

925ce1b

add whatsnew

b1c7f87

Fix detection to raise warning

656ec44

update wording

0b65a7a

DOC: GH21194

a1d966a

BUG: GH21194

5603bd2

TST: adding tz tests

1886cc3

msmarchena added 7 commits June 1, 2018 14:10

solving PEP8 errors

09348c6

DOC update

e2cc031

Merge remote-tracking branch 'upstream/master'

9cb6242

DOC update

d353ddb

DOC update

b421c2e

Merge branch 'master' of github.com:msmarchena/pandas into PR21194

0996c19

Doc update

1b398d9

jschendel reviewed Jun 1, 2018

View reviewed changes

Doc update

91a4547

jschendel added Datetime Datetime data dtype Indexing Related to indexing on series/frames, not to indexes themselves Dtype Conversions Unexpected or buggy dtype conversions labels Jun 1, 2018

jschendel added this to the 0.23.1 milestone Jun 1, 2018

mroeschke reviewed Jun 1, 2018

View reviewed changes

msmarchena added 3 commits June 4, 2018 18:31

parametrizing tests

f76d423

parametrizing tests

8cc507b

parametrizing tests

8441881

jreback requested changes Jun 4, 2018

View reviewed changes

jreback requested changes Jun 5, 2018

View reviewed changes

jreback modified the milestones: 0.23.1, 0.23.2 Jun 7, 2018

msmarchena added 6 commits June 25, 2018 12:07

Implementing pytest.mark.parametrize

d3cc807

Implementing pytest.mark.parametrize

8726d0e

Documentation update

e1cdfa5

Merge branch 'pr/12051' of https://github.com/msmarchena/pandas into …

246326d

…pr/12051

Merging

4c91628

Merge branch 'PR21194' of https://github.com/msmarchena/pandas into P…

79dbfe6

…R21194

jreback modified the milestones: 0.23.2, 0.23.3 Jun 26, 2018

jreback removed this from the 0.23.4 milestone Aug 2, 2018

Merge branch 'master' into PR21194

a5bc90b

TomAugspurger mentioned this pull request Aug 3, 2018

DataFrame.asof() : Timezone Awareness / Naivety comparison TypeError (incorrect) #21194

Closed

msmarchena closed this Aug 4, 2018

msmarchena deleted the PR21194 branch August 4, 2018 13:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

BUG: DataFrame.asof() : Timezone Awareness / Naivety comparison TypeError (incorrect) #21284

BUG: DataFrame.asof() : Timezone Awareness / Naivety comparison TypeError (incorrect) #21284

Uh oh!

msmarchena commented Jun 1, 2018

pep8speaks commented Jun 1, 2018 •

edited

Loading

jschendel Jun 1, 2018

msmarchena Jun 1, 2018

codecov bot commented Jun 1, 2018 •

edited

Loading

mroeschke Jun 1, 2018 •

edited

Loading

mroeschke Jun 1, 2018

jreback Jun 4, 2018

jreback Jun 4, 2018

jreback Jun 4, 2018

msmarchena Jun 5, 2018

jreback Jun 5, 2018

msmarchena Jun 5, 2018

msmarchena Jun 5, 2018

jreback Jun 5, 2018

jreback commented Jun 19, 2018

jreback commented Aug 1, 2018

Labels

7 participants

Uh oh!

BUG: DataFrame.asof() : Timezone Awareness / Naivety comparison TypeError (incorrect) #21284

BUG: DataFrame.asof() : Timezone Awareness / Naivety comparison TypeError (incorrect) #21284

Uh oh!

Conversation

msmarchena commented Jun 1, 2018

pep8speaks commented Jun 1, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comment last updated on August 03, 2018 at 14:02 Hours UTC

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jun 1, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

mroeschke Jun 1, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Jun 19, 2018

jreback commented Aug 1, 2018

Labels

7 participants

pep8speaks commented Jun 1, 2018 •

edited

Loading

codecov bot commented Jun 1, 2018 •

edited

Loading

mroeschke Jun 1, 2018 •

edited

Loading