Skip to content

Conversation

@jbrockmendel
Copy link
Member

cc @WillAyd the simple_frame path is kludgy, I expect fragile to non-numpy blocks. This just rips it out to simplify the code.

@jbrockmendel jbrockmendel requested a review from WillAyd as a code owner August 21, 2023 15:21
@lithomas1
Copy link
Contributor

Any perf impact?

Copy link
Member

@WillAyd WillAyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm if no regression. good to get rid of these branches

@mroeschke mroeschke added Refactor Internal refactoring of code IO JSON read_json, to_json, json_normalize labels Aug 21, 2023
@jbrockmendel
Copy link
Member Author

import numpy as np import pandas as pd arr = np.random.randn(10**4, 10) df = pd.DataFrame(arr) In [4]: %timeit df.to_json() 12.8 ms ± 1.72 ms per loop (mean ± std. dev. of 7 runs, 100 loops each) # <- main 12.4 ms ± 448 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) # <- PR In [5]: %timeit df.T.to_json() 12 ms ± 413 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) # <- main 19.2 ms ± 309 µs per loop (mean ± std. dev. of 7 runs, 10 loops each) # <- PR 

So for very-wide cases this takes a hit, but indistinguishable for normal cases.

@WillAyd
Copy link
Member

WillAyd commented Aug 21, 2023

I think that is fine. Not worth optimizing to_json for very wide frames at the ms level

@mroeschke mroeschke added this to the 2.2 milestone Aug 21, 2023
@mroeschke mroeschke merged commit 61e54c2 into pandas-dev:main Aug 21, 2023
@mroeschke
Copy link
Member

Thanks @jbrockmendel

@jbrockmendel jbrockmendel deleted the ref-json-simple branch August 21, 2023 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

IO JSON read_json, to_json, json_normalize Refactor Internal refactoring of code

4 participants