Questions tagged [benchmarking]
The benchmarking tag has no summary.
42 questions
5 votes
5 answers
722 views
How to test for performance regression?
I am working on a refactor on a certain package (I can give details if asked). The package involves a clever lazy evaluation of a sort of nested sequence of arithmetic operations. If the numerical ...
- 343
10 votes
7 answers
4k views
Does memoization skew benchmarks?
I'm writing a program that works with directed acyclic graphs and performs rewriting operations on them. Sometimes the graphs are small, sometimes they can be very large and grow as the program ...
- 621
3 votes
1 answer
292 views
Benchmarking C# code - results
I watch this YouTuber's mostly excellent videos on .NET features (let's call him Nick). He more often than not uses Benchmark.net to demonstrate the differences in performance of different approaches ...
- 18.6k
29 votes
5 answers
4k views
Benchmarking, why discard lowest time?
I've quite frequently seen benchmarks where the tester discarded the highest and the lowest time out of N runs. Discarding the highest time I understand; it's probably high because of some other ...
- 499
3 votes
1 answer
433 views
What is the expected performance of While loops using `array.pop()` assignment vs other methods
Recently I was asked to refactor some code that leverages JavaScript's array.reduce() method because other developers felt the code hard to read. While doing this I decided to play around with some ...
- 149
-3 votes
1 answer
214 views
Methods for reliably benchmarking own program's run time
I'm writing a basic password cracker in C as an introduction to multithreaded programming. I've implemented this already using a 'parallelization' approach that spins up a set number of threads and ...
- 109
2 votes
3 answers
5k views
Comparison of build times on various hardware; why non-linear results?
Why do the compile times not vary significantly between different era CPUs, even though disk (NVMe vs. HDD) and CPU benchmarks vary significantly in performance? Why does disabling hyperthreading ...
- 355
2 votes
0 answers
99 views
How to write disk benchmarking software?
I am coding a disk (HDD/SSD) benchmarking software. I have a few questions on how a disk should be benchmarked (regardless of programming language used). This is my pseudo code for testing sequential ...
- 175
3 votes
2 answers
2k views
How to benchmark software performance under same conditions (RAM, CPU caching etc)?
I am writing a software that splits big files into smaller files and I have coded several solutions. I am measuring the execution time of each of my solutions. (with threads, go routines, MPI etc) ...
- 247
-3 votes
1 answer
92 views
Handling different versions of a program and benchmarking with OO
Note: I'm not sure if the way I phrase the question initially will fit 100% on this board, so please help me to focus on the main point. Scenario: You develop an algorithm or a neural network that ...
- 111
2 votes
1 answer
334 views
How do I triage a website effectively?
I've inherited responsibilities for maintaining a section of enterprise intranet, and my manager has tasked me with, "Clean up the code, please. We're probably going to get lots of change requests ...
- 217
3 votes
1 answer
350 views
How to compare C and CUDA performance fairly?
I have developed a CUDA code that is a reimplementation of a (CPU only) C code. I would like to benchmark the performance of these two programs. I am wondering how to do benchmarking well in this ...
- 593
1 vote
2 answers
1k views
Can I use statistical methods to compare benchmark performance figures?
Imagine I am working on a C compiler. I have made some changes to the compiler's code, and am interested to know what effect these changes have on the performance of generated binaries. I have a ...
- 167
4 votes
4 answers
1k views
How to manage code experiments meant to stay permanently with git?
I have an asm/C code which implements some image filters. The objective of this project is experimentation with different implementations, benchmarking, plotting and reporting data to write a paper ...
- 143
5 votes
2 answers
139 views
Estimate if benchmark sampling difference is significative
In order to estimate the fastest function in various cases in a non fully deterministic context, I'm running an experiment calling one or the other at random and recording the duration of the ...
- 1,454