Skip to content

Conversation

@mroeschke
Copy link
Member

  • tests added / passed
  • Ensure all linting tests pass, see here for how to run them
  • whatsnew entry

This also starts to add a shared aggregation function (sum) that can shared between rolling/groupby/DataFrame when using the numba engine.

df = pd.DataFrame(np.ones((10000, 1000))) roll = df.rolling(10) roll.sum(engine="numba", engine_kwargs={"nopython": True, "nogil": True, "parallel": True}) %timeit roll.sum(engine="numba", engine_kwargs={"nopython": True, "nogil": True, "parallel": True}) 211 ms ± 12.2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) <- PR 424 ms ± 9.23 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) <- master 
@mroeschke mroeschke added Performance Memory or execution speed performance Window rolling, ewma, expanding labels Oct 25, 2021
@mroeschke mroeschke added this to the 1.4 milestone Oct 25, 2021
@jreback jreback merged commit 5d520c6 into pandas-dev:master Oct 28, 2021
@jreback
Copy link
Contributor

jreback commented Oct 28, 2021

great, can you make sure that we have sufficient benchmarks

e.g.

  • cython vs numba parallel=True & False
  • small number of cols vs large number
@mroeschke mroeschke deleted the enh/rolling_numba_sum branch October 29, 2021 02:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Performance Memory or execution speed performance Window rolling, ewma, expanding

2 participants