Skip to main content
Software Engineering

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

7
  • 24
    This answer contains a great truth: benchmarking beats lots of theory and speculation. Commented Sep 3, 2020 at 7:43
  • 6
    @qwr An adage also known as : "One experiment is worth a thousand expert opinions" Commented Sep 4, 2020 at 13:25
  • 5
    @qwr: another great truth is that benchmarking is terribly difficult. In particular when developing general purpose solutions such that you don't know the problem size and specifics in advance (and sometimes even have no idea of realistic problems). Some understanding of the trends in behavior is not luxury. Commented Sep 9, 2020 at 9:49
  • 3
    @YvesDaoust, this is especially true for parallelized algorithms, where it is far harder to set up "controlled" benchmark experiments. Commented Sep 10, 2020 at 10:29
  • @taciteloquence the few things I remember from my university days was a lesson about what used to be simply called 'network computing': unlike single-physical-CPU systems, they're chaotic and totally unpredictable. There is no such thing as a 'controlled benchmark experiment' — you can establish a few baselines, sure, but real life will always be different from what you can assemble in a lab. Commented Sep 10, 2020 at 11:14