Skip to content

IntervalArray.shift and missing value handling #22428

@TomAugspurger

Description

@TomAugspurger

Followup to #22387

The default implementation of shift fails when dtype.na_dtype can't be stored in a dtype array (e.g. int can't hold na).

In [24]: idx = IntervalArray.from_breaks(range(10)) In [25]: idx.shift() --------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-25-1b2c2192e1e6> in <module>() ----> 1 idx.shift() ~/sandbox/pandas/pandas/core/arrays/base.py in shift(self, periods) 422 return self.copy() 423 empty = self._from_sequence([self.dtype.na_value] * abs(periods), --> 424 dtype=self.dtype) 425 if periods > 0: 426 a = empty ~/sandbox/pandas/pandas/core/arrays/interval.py in _from_sequence(cls, scalars, dtype, copy) 193 @classmethod 194 def _from_sequence(cls, scalars, dtype=None, copy=False): --> 195 return cls(scalars, dtype=dtype, copy=copy) 196 197 @classmethod ~/sandbox/pandas/pandas/core/arrays/interval.py in __new__(cls, data, closed, dtype, copy, fastpath, verify_integrity) 138 139 return cls._simple_new(left, right, closed, copy=copy, dtype=dtype, --> 140 verify_integrity=verify_integrity) 141 142 @classmethod ~/sandbox/pandas/pandas/core/arrays/interval.py in _simple_new(cls, left, right, closed, copy, dtype, verify_integrity) 156 raise TypeError(msg.format(dtype=dtype)) 157 elif dtype.subtype is not None: --> 158 left = left.astype(dtype.subtype) 159 right = right.astype(dtype.subtype) 160 ~/sandbox/pandas/pandas/core/indexes/numeric.py in astype(self, dtype, copy) 316 elif is_integer_dtype(dtype) and self.hasnans: 317 # GH 13149 --> 318 raise ValueError('Cannot convert NA to integer') 319 return super(Float64Index, self).astype(dtype, copy=copy) 320 ValueError: Cannot convert NA to integer

Perhaps we can investigate using our IntegerNA extension array for the storage of int-dtyped IntervalArrays?

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugExtensionArrayExtending pandas with custom dtypes or arrays.IntervalInterval data typeMissing-datanp.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions