Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
ec577d7
set self.observed=True for .sum() and .count() so that reindex can be…
smithto1 Jul 11, 2020
586b1bd
wrote test for .sum() and .count() exception handling
smithto1 Jul 11, 2020
ec4bd28
black
smithto1 Jul 11, 2020
eed48cb
updated pivot tests
smithto1 Jul 11, 2020
88d8a14
whatsnew
smithto1 Jul 11, 2020
940b32e
comments
smithto1 Jul 11, 2020
3e39f1a
black
smithto1 Jul 11, 2020
bd4c4fe
.sum() and .count() create inner groupby with observed=True, and then…
smithto1 Jul 14, 2020
fcbb58b
modify self.observed with a context manager
smithto1 Jul 15, 2020
5f89c17
added comments
smithto1 Jul 15, 2020
7ed0982
merging master
smithto1 Jul 15, 2020
9da5b18
Fixed import order that caused Linting error
smithto1 Jul 15, 2020
534d60f
addressing comments
smithto1 Jul 16, 2020
b323c9c
merging master
smithto1 Jul 16, 2020
d2abd4d
fixing whatsnew comment
smithto1 Jul 16, 2020
361098a
ammend comment
smithto1 Jul 16, 2020
9879199
ammend comment
smithto1 Jul 16, 2020
39ee6d0
addressing PR comments
smithto1 Jul 16, 2020
fc1a846
ammend comment
smithto1 Jul 16, 2020
f774288
amend comment
smithto1 Jul 17, 2020
1986f51
amend comment (just to rerun tests
smithto1 Jul 17, 2020
c4c1a92
ammend comment to start new tests (tests failing due to some flakey t…
smithto1 Jul 17, 2020
7093c91
merging master
smithto1 Jul 18, 2020
5f8ba0e
amend comment to kick off tests
smithto1 Jul 18, 2020
49b5fc8
ammend test to kick off tests
smithto1 Jul 18, 2020
0454e3a
merging master
smithto1 Jul 26, 2020
28794e3
fixed merge
smithto1 Jul 26, 2020
f616d5a
merge fix
smithto1 Jul 26, 2020
80d4b7d
udpate comment to kick off tests
smithto1 Jul 27, 2020
442ad1d
amend comment to restart checks
smithto1 Aug 1, 2020
be8b78c
Merge remote-tracking branch 'upstream/master' into issue31422-v7
smithto1 Aug 5, 2020
ed99b60
doc updates
smithto1 Aug 5, 2020
528ff7e
restart tests
smithto1 Aug 5, 2020
b2a331f
Merge remote-tracking branch 'upstream/master' into issue31422-v7
smithto1 Aug 6, 2020
cb010fa
Merge branch 'master' into issue31422-v7
jreback Aug 7, 2020
d6ebe10
merge
smithto1 Aug 7, 2020
f575687
merge
smithto1 Aug 7, 2020
File filter

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
addressing comments
  • Loading branch information
smithto1 committed Jul 16, 2020
commit 534d60fcd628438588df6d9173f6db93b0883395
6 changes: 2 additions & 4 deletions doc/source/whatsnew/v1.1.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1081,8 +1081,7 @@ Groupby/resample/rolling
- Bug in :meth:`DataFrame.ewm.cov` was throwing ``AssertionError`` for :class:`MultiIndex` inputs (:issue:`34440`)
- Bug in :meth:`core.groupby.DataFrameGroupBy.transform` when ``func='nunique'`` and columns are of type ``datetime64``, the result would also be of type ``datetime64`` instead of ``int64`` (:issue:`35109`)
- Bug in :meth:'DataFrameGroupBy.first' and :meth:'DataFrameGroupBy.last' that would raise an unnecessary ``ValueError`` when grouping on multiple ``Categoricals`` (:issue:`34951`)
- Bug in :meth:`DataFrameGroupBy.count` was returning ``NaN`` for missing categories when grouped on multiple ``Categoricals``. Now returning ``0`` (:issue:`35028`)
- Bug in :meth:`DataFrameGroupBy.sum` and :meth:`SeriesGroupBy.sum` was reutrning ``NaN`` for missing categories when grouped on multiple ``Categorials``. Now returning ``0`` (:issue:`31422`)
- Bug in :meth:`DataFrameGroupBy.count` and :meth:`GroupBy.sum` returning ``NaN`` for missing categories when grouped on multiple ``Categoricals``. Now returning ``0`` (:issue:`35028`)


Reshaping
Expand Down Expand Up @@ -1118,8 +1117,7 @@ Reshaping
- Bug in :meth:`Series.where` with an empty Series and empty ``cond`` having non-bool dtype (:issue:`34592`)
- Fixed regression where :meth:`DataFrame.apply` would raise ``ValueError`` for elements whth ``S`` dtype (:issue:`34529`)
- Bug in :meth:`DataFrame.append` leading to sorting columns even when ``sort=False`` is specified (:issue:`35092`)
- Bug in :meth:`DataFrame.pivot_table` with ``aggfunc='count'``, was returning ``NaN`` for missing categories when pivoted on a ``Categorical``. Now returning ``0`` (:issue:`35028`)
- Bug in :meth:`DataFrame.pivot_table` with ``aggfunc='sum'``, was reutrning ``NaN`` for missing categories when pivoted on a ``Categorical``. Now returning ``0`` (:issue:`31422`)
- Bug in :meth:`DataFrame.pivot_table` with ``aggfunc='count'`` or ``aggfunc='sum'`` returning ``NaN`` for missing categories when pivoted on a ``Categorical``. Now returning ``0`` (:issue:`31422`)

Sparse
^^^^^^
Expand Down
3 changes: 1 addition & 2 deletions pandas/core/groupby/generic.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,6 @@
GroupBy,
_agg_template,
_apply_docs,
_observed_is_true,
_transform_template,
get_groupby,
)
Expand Down Expand Up @@ -1833,7 +1832,7 @@ def count(self):
# _wrap_agged_blocks() returns. We set self.observed=True for the call to
# _wrap_agged_blocks() to avoid it reindexing. We then reindex below with
# fill_value=0. See GH 35028
with _observed_is_true(self):
with com.temp_setattr(self, "observed", True):
result = self._wrap_agged_blocks(blocks, items=data.items)

return self._reindex_output(result, fill_value=0)
Expand Down
13 changes: 1 addition & 12 deletions pandas/core/groupby/groupby.py
Original file line number Diff line number Diff line change
Expand Up @@ -464,17 +464,6 @@ def _group_selection_context(groupby):
groupby._reset_group_selection()


@contextmanager
def _observed_is_true(groupby):
"""
Set GroupBy.observed to True, then restore to original value
"""
original_value = groupby.observed
groupby.observed = True
yield groupby
groupby.observed = original_value


_KeysArgType = Union[
Hashable,
List[Hashable],
Expand Down Expand Up @@ -1548,7 +1537,7 @@ def sum(self, numeric_only: bool = True, min_count: int = 0):
# _agg_general() returns. We set self.observed=True for the call to
# _agg_general() to avoid it reindexing. We then reindex below with
# fill_value=0. See GH 31422
with _observed_is_true(self):
with com.temp_setattr(self, "observed", True):
result = self._agg_general(
numeric_only=numeric_only,
min_count=min_count,
Expand Down