API: make CategoricalIndex._concat consistent with pd.concat #41626
Description
Index._concat (used by Index.append) is thin wrapper around concat_compat. It is overriden by CategoricalIndex so that CategoricalDtype is retained more often than it is in concat_compat. We should make these match.
If we just rip CategoricalIndex._concat, we break 6 tests, all of which boil down to:
def test_append_category_objects(self, ci): # with objects result = ci.append(Index(["c", "a"])) expected = CategoricalIndex(list("aabbcaca"), categories=ci.categories) > tm.assert_index_equal(result, expected, exact=True) If we go the other way and change concat_compat, we break 6 different tests, all of which involve all-empty arrays or arrays that can be losslessly cast to the Categorical's dtype, e.g (edited for legibility)
def test_concat_empty_series_dtype_category_with_array(self): # GH#18515 left = Series(np.array([]), dtype="category") right = Series(dtype="float64") result = concat([left, right]) > assert result.dtype == "float64" def test_concat_categorical_coercion(self): # GH 13524 # category + not-category => not-category s1 = Series([1, 2, np.nan], dtype="category") s2 = Series([2, 1, 2]) exp = Series([1, 2, np.nan, 2, 1, 2], dtype="object") > tm.assert_series_equal(pd.concat([s1, s2], ignore_index=True), exp) E AssertionError: Attributes of Series are different E E Attribute "dtype" are different E [left]: CategoricalDtype(categories=[1, 2], ordered=False) E [right]: object Changing concat_compat results in much more convenient behavior, but it is textbook "values-dependent behavior" that in general we want to avoid (cc @jorisvandenbossche)