Skip to content

BUG: Series[int].loc setitem with Series[int] results in Series[float] #41644

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jun 8, 2021
Merged

BUG: Series[int].loc setitem with Series[int] results in Series[float] #41644

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
c9b617b
BUG: Series[int].loc setitem with Series[int] results in Series[float]
simonjayhawkins May 24, 2021
46ee828
pre-commit fix
simonjayhawkins May 24, 2021
776c144
Merge remote-tracking branch 'upstream/master' into can-hold-element
simonjayhawkins Jun 1, 2021
f8f9bb5
remove backport fix
simonjayhawkins Jun 1, 2021
0ab9c47
Merge remote-tracking branch 'upstream/master' into can-hold-element
simonjayhawkins Jun 4, 2021
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions pandas/core/dtypes/cast.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2218,6 +2218,8 @@ def can_hold_element(arr: ArrayLike, element: Any) -> bool:
if dtype.kind in ["i", "u"]:
if tipo is not None:
if tipo.kind not in ["i", "u"]:
if is_float(element) and element.is_integer():
return True
# Anything other than integer we cannot hold
return False
elif dtype.itemsize < tipo.itemsize:
Expand Down
8 changes: 0 additions & 8 deletions pandas/core/internals/blocks.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -781,14 +781,6 @@ def _replace_list(
# so un-tile here
return self.replace(src_list, dest_list[0], inplace, regex)

# https://github.com/pandas-dev/pandas/issues/40371
# the following pairs check code caused a regression so we catch that case here
# until the issue is fixed properly in can_hold_element

# error: "Iterable[Any]" has no attribute "tolist"
if hasattr(src_list, "tolist"):
src_list = src_list.tolist() # type: ignore[attr-defined]

# Exclude anything that we know we won't contain
pairs = [
(x, y) for x, y in zip(src_list, dest_list) if self._can_hold_element(x)
Expand Down
2 changes: 1 addition & 1 deletion pandas/tests/series/indexing/test_setitem.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ def test_setitem_series_object_dtype(self, indexer, ser_index):
expected = Series([Series([42], index=[ser_index]), 0], dtype="object")
tm.assert_series_equal(ser, expected)

@pytest.mark.parametrize("index, exp_value", [(0, 42.0), (1, np.nan)])
@pytest.mark.parametrize("index, exp_value", [(0, 42), (1, np.nan)])
def test_setitem_series(self, index, exp_value):
Copy link
Member

@jbrockmendel jbrockmendel May 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems weird. wouldnt we expect an object-dtype Series with ser.loc[0] itself being a Series?
Copy link
Member Author

@simonjayhawkins simonjayhawkins May 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's unrelated to the changes here. so will not be changing the indexing code in this PR. This PR is a followon to #40555 to change can_hold_element, which I'm uncomfortable doing on 1.2.x since we may not have another release after the next one.
Copy link
Member

@jbrockmendel jbrockmendel May 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. cc @phofl thoughts on desired behavior?
Copy link
Member

@phofl phofl May 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the behavior after this pr is correct.

ser.loc[0] = Series([42], index=[0])

This assigns 42 to the first element in the Series, so would not expect a type cast here
Copy link
Member

@jbrockmendel jbrockmendel May 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so you expect to get the same result from ser.loc[0] = ser2 as you would from ser.loc[[0]] = ser2?
Copy link
Contributor

@jreback jreback Jun 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did this resolve? (is this a separate patch / issue s well)?
Copy link
Member Author

@simonjayhawkins simonjayhawkins Jun 5, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need a separate issue/patch if any other changes. @jbrockmendel @phofl

if another patch is done, this PR is not urgent as it becomes just a refactor.

if another patch is not done, we should merge this before 1.3.
Copy link
Member Author

@simonjayhawkins simonjayhawkins Jun 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jbrockmendel @phofl is there a plan to change the return type. otherwise we should probably merge this for 1.3

waiting and we would need a release note and decide on backport (which i'd be -1 on, for the same reason I didn't change can_hold_element in 1.2.5)
Copy link
Member

@phofl phofl Jun 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm too. Even if we agree that we should return something else, the int result is more consistent than the float result
Copy link
Member

@jbrockmendel jbrockmendel Jun 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The correct-behavior question may be related to #16864
# GH#38303
ser = Series([0, 0])
Expand Down