ENH: Make categories setitem error more readable #46646
Description
Pandas version checks
-
I have checked that this issue has not already been reported.
-
I have confirmed this bug exists on the latest version of pandas.
-
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
>>> import pandas as pd >>> ps = pd.Series([1, 2, 3], dtype="category") # this is the current behavior >>> ps[0] = 5 Traceback (most recent call last): File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/series.py", line 1085, in __setitem__ self._set_with_engine(key, value) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/series.py", line 1149, in _set_with_engine self._mgr.setitem_inplace(loc, value) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/internals/base.py", line 190, in setitem_inplace arr[indexer] = value File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/arrays/_mixins.py", line 249, in __setitem__ value = self._validate_setitem_value(value) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/arrays/categorical.py", line 1457, in _validate_setitem_value return self._validate_scalar(value) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/arrays/categorical.py", line 1484, in _validate_scalar raise TypeError( TypeError: Cannot setitem on a Categorical with a new category (5), set the categories first During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/series.py", line 1140, in __setitem__ self._set_with(key, value) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/series.py", line 1167, in _set_with self._set_labels(key, value) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/series.py", line 1179, in _set_labels self._set_values(indexer, value) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/series.py", line 1185, in _set_values self._mgr = self._mgr.setitem(indexer=key, value=value) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/internals/managers.py", line 337, in setitem return self.apply("setitem", indexer=indexer, value=value) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/internals/managers.py", line 304, in apply applied = getattr(b, f)(**kwargs) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/internals/blocks.py", line 1604, in setitem self.values[indexer] = value File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/arrays/_mixins.py", line 249, in __setitem__ value = self._validate_setitem_value(value) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/arrays/categorical.py", line 1457, in _validate_setitem_value return self._validate_scalar(value) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/arrays/categorical.py", line 1484, in _validate_scalar raise TypeError( TypeError: Cannot setitem on a Categorical with a new category (5), set the categories firstIssue Description
_validate_scalar in categorical.py is propagating an exception that could often lead to confusing error stack-trace because of the previous errors that would have occured. So instead of having the following code:
pandas/pandas/core/arrays/categorical.py
Lines 1474 to 1477 in c8cbe19
If it is replace with:
raise TypeError( "Cannot setitem on a Categorical with a new " f"category ({fill_value}), set the categories first" ) from NoneThe error stack-trace will be cleaner like as follows:
Expected Behavior
# This is much cleaner as we don't have the previous exceptions stack-trace. Which is actually irrelevant at this point. >>> ps[0] = 5 Traceback (most recent call last): File "<console>", line 1, in <module> File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/series.py", line 1140, in __setitem__ self._set_with(key, value) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/series.py", line 1167, in _set_with self._set_labels(key, value) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/series.py", line 1179, in _set_labels self._set_values(indexer, value) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/series.py", line 1185, in _set_values self._mgr = self._mgr.setitem(indexer=key, value=value) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/internals/managers.py", line 337, in setitem return self.apply("setitem", indexer=indexer, value=value) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/internals/managers.py", line 304, in apply applied = getattr(b, f)(**kwargs) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/internals/blocks.py", line 1604, in setitem self.values[indexer] = value File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/arrays/_mixins.py", line 249, in __setitem__ value = self._validate_setitem_value(value) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/arrays/categorical.py", line 1457, in _validate_setitem_value return self._validate_scalar(value) File "/nvme/0/pgali/envs/cudfdev/lib/python3.8/site-packages/pandas/core/arrays/categorical.py", line 1484, in _validate_scalar raise TypeError( TypeError: Cannot setitem on a Categorical with a new category (5), set the categories firstInstalled Versions
Replace this line with the output of pd.show_versions()