BUG: .get_indexer_non_unique() must return an array of ints (#44084) #44404
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
…ev#44084) GH#44084 boils down to the following. According to the docs `.get_indexer_non_unique()` is supposed to return "integers from 0 to n - 1 indicating that the index at these positions matches the corresponding target values". However, for an index that is non unique and non monotonic it returns a boolean mask. That is because it uses `.get_loc()` which for non unique, non monotonic indexes returns a boolean mask. This patch catches that case and converts the boolean mask from `.get_loc()` into the corresponding array of integers if the index is not unique and not monotonic.
phofl requested changes Nov 12, 2021
| | ||
| def test_get_index_non_unique_non_monotonic(self): | ||
| # GH#44084 | ||
| index = IntervalIndex.from_tuples( |
Member
There was a problem hiding this comment.
Could you add the MultiIndex cases
On Windows `np.array([1, 3])` is obviously int32 and thus the comparison to the int64 array fails due to dtype mismatch.
Sometimes the world out there is a bit more complicated than what you have on your cozy desktop :)
phofl approved these changes Nov 12, 2021
phofl added Indexing 
Contributor
| thanks @johannes-mueller do we have sufficient testing on this for other index types aside from those explicityly tested here? if not would take a PR for that |
4 tasks
nickleus27 pushed a commit to nickleus27/pandas that referenced this pull request Nov 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
GH#44084 boils down to the following.
According to the docs
.get_indexer_non_unique()is supposed to return"integers from 0 to n - 1 indicating that the index at these positions matches
the corresponding target values". However, for an index that is non unique and
non monotonic it returns a boolean mask. That is because it uses
.get_loc()which for non unique, non monotonic indexes returns a boolean mask.
This patch catches that case and converts the boolean mask from
.get_loc()into the corresponding array of integers if the index is not unique and not
monotonic.