Skip to content

Conversation

@mroeschke
Copy link
Member

@mroeschke mroeschke commented Aug 25, 2022

@mroeschke mroeschke added Output-Formatting __repr__ of pandas objects, to_string Dtype Conversions Unexpected or buggy dtype conversions Timezones Timezone data dtype Arrow pyarrow functionality labels Aug 25, 2022
@mroeschke mroeschke added this to the 1.5 milestone Aug 25, 2022
def kind(self) -> str:
return "M"

assert not com.is_datetime64tz_dtype(NotTZDtype())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could also check needs_i8_conversion here

@mroeschke
Copy link
Member Author

@jbrockmendel don't know if you've ever hit this while working on EA.argmin/max/sort. For some reason (seemly) unrelated type's, in this case timedelta, argmin/max/sort fail. I got a similar error when trying to define ArrowExtensionArray._values_for_argsort, but ended up removing it since I just could define ArrowExtensionArray.argmin/max/sort instead.

https://github.com/pandas-dev/pandas/runs/8027023184?check_suite_focus=true

2022-08-26T00:21:38.2857760Z =================================== FAILURES =================================== 2022-08-26T00:21:38.2858430Z _________________ TestIndexReductions.test_minmax_timedelta64 __________________ 2022-08-26T00:21:38.2860770Z [gw0] darwin -- Python 3.9.13 /Users/runner/micromamba/envs/test/bin/python 2022-08-26T00:21:38.2861240Z 2022-08-26T00:21:38.2861500Z obj = array([ 86400000000000, 172800000000000, 259200000000000], 2022-08-26T00:21:38.2862360Z dtype='timedelta64[ns]') 2022-08-26T00:21:38.2871110Z method = 'argmin', args = (), kwds = {'axis': None, 'out': None} 2022-08-26T00:21:38.2878500Z bound = <built-in method argmin of numpy.ndarray object at 0x178dea3f0> 2022-08-26T00:21:38.2879060Z 2022-08-26T00:21:38.2879260Z def _wrapfunc(obj, method, *args, **kwds): 2022-08-26T00:21:38.2892110Z bound = getattr(obj, method, None) 2022-08-26T00:21:38.2892850Z if bound is None: 2022-08-26T00:21:38.2893420Z return _wrapit(obj, method, *args, **kwds) 2022-08-26T00:21:38.2894830Z 2022-08-26T00:21:38.2896420Z try: 2022-08-26T00:21:38.2896970Z > return bound(*args, **kwds) 2022-08-26T00:21:38.2898150Z E TypeError: Cannot cast array data from dtype('<m8[ns]') to dtype('<m8[us]') according to the rule 'safe' 2022-08-26T00:21:38.2898680Z 2022-08-26T00:21:38.2899440Z ../../../micromamba/envs/test/lib/python3.9/site-packages/numpy/core/fromnumeric.py:57: TypeError 2022-08-26T00:21:38.2899890Z 2022-08-26T00:21:38.2900210Z During handling of the above exception, another exception occurred: 2022-08-26T00:21:38.2900610Z 2022-08-26T00:21:38.2901140Z self = <pandas.tests.reductions.test_reductions.TestIndexReductions object at 0x15efe5d00> 2022-08-26T00:21:38.2901860Z 2022-08-26T00:21:38.2902170Z def test_minmax_timedelta64(self): 2022-08-26T00:21:38.2902670Z 2022-08-26T00:21:38.2903110Z # monotonic 2022-08-26T00:21:38.2903720Z idx1 = TimedeltaIndex(["1 days", "2 days", "3 days"]) 2022-08-26T00:21:38.2905060Z assert idx1.is_monotonic_increasing 2022-08-26T00:21:38.2905630Z 2022-08-26T00:21:38.2906250Z # non-monotonic 2022-08-26T00:21:38.2906710Z idx2 = TimedeltaIndex(["1 days", np.nan, "3 days", "NaT"]) 2022-08-26T00:21:38.2907330Z assert not idx2.is_monotonic_increasing 2022-08-26T00:21:38.2907860Z 2022-08-26T00:21:38.2909520Z for idx in [idx1, idx2]: 2022-08-26T00:21:38.2911780Z assert idx.min() == Timedelta("1 days") 2022-08-26T00:21:38.2912570Z assert idx.max() == Timedelta("3 days") 2022-08-26T00:21:38.2913080Z > assert idx.argmin() == 0 2022-08-26T00:21:38.2913940Z 2022-08-26T00:21:38.2914320Z pandas/tests/reductions/test_reductions.py:266: 2022-08-26T00:21:38.2915080Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2022-08-26T00:21:38.2915810Z pandas/core/indexes/base.py:7175: in argmin 2022-08-26T00:21:38.2916930Z return super().argmin(skipna=skipna) 2022-08-26T00:21:38.2917250Z pandas/core/base.py:713: in argmin 2022-08-26T00:21:38.2917510Z return delegate.argmin() 2022-08-26T00:21:38.2917860Z pandas/core/arrays/_mixins.py:200: in argmin 2022-08-26T00:21:38.2918570Z return nargminmax(self, "argmin", axis=axis) 2022-08-26T00:21:38.2919230Z pandas/core/sorting.py:480: in nargminmax 2022-08-26T00:21:38.2920470Z return _nanargminmax(arr_values, mask, func) 2022-08-26T00:21:38.2920970Z pandas/core/sorting.py:491: in _nanargminmax 2022-08-26T00:21:38.2921390Z return non_nan_idx[func(non_nans)] 2022-08-26T00:21:38.2921820Z <__array_function__ internals>:180: in argmin 2022-08-26T00:21:38.2922200Z ??? 2022-08-26T00:21:38.2923120Z ../../../micromamba/envs/test/lib/python3.9/site-packages/numpy/core/fromnumeric.py:1312: in argmin 2022-08-26T00:21:38.2923920Z return _wrapfunc(a, 'argmin', axis=axis, out=out, **kwds) 2022-08-26T00:21:38.2924780Z ../../../micromamba/envs/test/lib/python3.9/site-packages/numpy/core/fromnumeric.py:66: in _wrapfunc 2022-08-26T00:21:38.2926060Z return _wrapit(obj, method, *args, **kwds) 2022-08-26T00:21:38.2927600Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2022-08-26T00:21:38.2927880Z 2022-08-26T00:21:38.2928170Z obj = array([ 86400000000000, 172800000000000, 259200000000000], 2022-08-26T00:21:38.2928850Z dtype='timedelta64[ns]') 2022-08-26T00:21:38.2929530Z method = 'argmin', args = (), kwds = {'axis': None, 'out': None} 2022-08-26T00:21:38.2931450Z wrap = <built-in method __array_wrap__ of numpy.ndarray object at 0x178dea3f0> 2022-08-26T00:21:38.2935490Z 2022-08-26T00:21:38.2935730Z def _wrapit(obj, method, *args, **kwds): 2022-08-26T00:21:38.2936160Z try: 2022-08-26T00:21:38.2936550Z wrap = obj.__array_wrap__ 2022-08-26T00:21:38.2941120Z except AttributeError: 2022-08-26T00:21:38.2957840Z wrap = None 2022-08-26T00:21:38.2958820Z > result = getattr(asarray(obj), method)(*args, **kwds) 2022-08-26T00:21:38.2962350Z E TypeError: Cannot cast array data from dtype('<m8[ns]') to dtype('<m8[us]') according to the rule 'safe' 2022-08-26T00:21:38.2962810Z 2022-08-26T00:21:38.2963520Z ../../../micromamba/envs/test/lib/python3.9/site-packages/numpy/core/fromnumeric.py:43: TypeError 
@jbrockmendel
Copy link
Member

i dont think ive seen that, no. is it flaky?

@jbrockmendel
Copy link
Member

[finger slipped on the wrong button]

@mroeschke
Copy link
Member Author

mroeschke commented Sep 8, 2022

i dont think ive seen that, no. is it flaky?

I don't think so since it appears on some specific builds. I've tried installing those environments on my laptop but can't replicating them.

I'll keep poking around. I suspect it has to do with some statefulness.

@mroeschke
Copy link
Member Author

mroeschke commented Sep 14, 2022

Noting if worked on the future: Disabling the pyarrow timedelta types in the tests 21d152c fixed the seemingly unrelated timedelta errors in #48258 (comment), so there may be some odd interaction between then for pyarrow version 5, 6, 7

@mroeschke mroeschke changed the title BUG: is_datetime64tz_dtype shortcut only for DatetimeTZDtype TST: Series repr with pyarrow date32 type Sep 14, 2022
@mroeschke mroeschke marked this pull request as draft September 15, 2022 00:37
@mroeschke mroeschke removed this from the 1.5 milestone Sep 15, 2022
@mroeschke
Copy link
Member Author

May revisit this later, but since the original issue has been address by another PR, closing

@mroeschke mroeschke closed this Sep 20, 2022
@mroeschke mroeschke deleted the bug/is_dttz_dtype branch September 20, 2022 22:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Arrow pyarrow functionality Dtype Conversions Unexpected or buggy dtype conversions Output-Formatting __repr__ of pandas objects, to_string Timezones Timezone data dtype

2 participants