BUG: GroupBy.size created by TimeGrouper raises AttributeError

[sinhrks]

Related to #7453 2nd issue:

• GroupBy.size created by TimeGrouper raises AttributeError

[jreback]

why are you not using value_counts? this should be very similar (if not call irectly), BaseGrouper.size

[sinhrks]

There are 2 reasons.

• value_counts cannot fulfill intermediate timestamps which is not included in the group key. For example, if the TimeGrouper categorize [2011-01-01, 2011-01-02, 2011-01-04] with daily frequency, the resulted group keys must be [2011-01-01, 2011-01-02, 2011-01-03, 2011-01-04]
• BinGrouper doesn't retain original Index to be directly passed to value_counts. To use value_counts, it must be created using similar logic.

indices = self.indices labels = [] for k, v in compat.iteritems(indices): labels.extend([k] * len(v)) value_counts(labels) 

[jreback]

ok can u add a bench for size in that case?
for both a regular groupby and TimeGrouper?

[sinhrks]

OK, added benchmarks.

[jreback]

can I post benchmarks for those 2? thxs

[sinhrks]

Here it is. Cannot measure base performance of groupby_dt_timegrouper_size because of the bug, but it looks not slow comparing to groupby_dt_size

------------------------------------------------------------------------------- Test name | head[ms] | base[ms] | ratio | ------------------------------------------------------------------------------- groupby_dt_size | 42.2956 | 55.4504 | 0.7628 | groupby_dt_timegrouper_size | 34.2657 | ------- | ------ | 

[jreback]

run then with -Hto test current perf (with your fix) with comparable samples (or do in ipython).

[sinhrks]

Yep, current perf of groupby_dt_timegrouper_size is attached above.