Skip to content

Add csimdv to count benchmark#55

Open
juliusgeo wants to merge 2 commits intomedialab:masterfrom
juliusgeo:add_csimdv_example
Open

Add csimdv to count benchmark#55
juliusgeo wants to merge 2 commits intomedialab:masterfrom
juliusgeo:add_csimdv_example

Conversation

@juliusgeo
Copy link

@juliusgeo juliusgeo commented Mar 17, 2026

Adds csimdv to benchmark file.
On my aarch64 machine, it works without any rustflags. On x86_64 (Windows), I had to provide RUSTFLAGS="target-cpu=native".

@Yomguithereal
Copy link
Member

First, on my end I need RUSTFLAGS="-C target-feature=+pclmulqdq,+avx2" because I don't have avx512f.

@Yomguithereal
Copy link
Member

Then, my functional tests crash because your buffer here: https://github.com/juliusgeo/csimdv-rs/blob/master/src/constants.rs#L3 is too short for real-life CSV data (any file containing raw text in a cell, like press articles, will overflow).

@Yomguithereal
Copy link
Member

Now with the files I can test (the ones which don't overflow the buffer), c-simd-v is consistently slower than the copying reader. Let me try an aggregate stats better. I must also find a CPU with AVX-512, since I am only benchmarking AVX2 here.

@juliusgeo
Copy link
Author

juliusgeo commented Mar 18, 2026

@Yomguithereal I bumped the buffer size in csimdv to 1mb, which should (hopefully) be large enough to accommodate real world files. I also realized that in the benchmark I was not only counting the number of rows in the CSV file, but also the length of each cell in the rows. I ran the benchmarks from this repo on the examples from the csimdv repo, and the results align much better after that change. Seeing roughly the same speed from zero-copy and csimdv on aarch64, will have to test later on AVX-512.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants