4
\$\begingroup\$

Intro

(See the next iteration A tiny Java framework for gathering running time statistics - Take II.)

I have this tiny Java framework that allows users to gather running time statistics of a piece of code. It gathers:

  • minimum running time of a Runnable,
  • maximum running time,
  • mean time,
  • median time,
  • standard deviation.

Code


package io.github.coderodde.statistics.run; import java.text.NumberFormat; /** * This class encapsulates the run statistics. * * @author Rodion "rodde" Efremov */ public final class RunStatistics { private final long minimumDuration; private final long maximumDuration; private final double meanDuration; private final double medianDuration; private final double standardDeviation; RunStatistics(final long minimumDuration, final long maximumDuration, final double meanDuration, final double medianDuration, final double standardDeviation) { this.minimumDuration = minimumDuration; this.maximumDuration = maximumDuration; this.meanDuration = meanDuration; this.medianDuration = medianDuration; this.standardDeviation = standardDeviation; } public long getMinimumDuration() { return minimumDuration; } public long getMaximumDuration() { return maximumDuration; } public double getMeanDuration() { return meanDuration; } public double getMedianDuration() { return medianDuration; } public double getStandardDeviation() { return standardDeviation; } @Override public String toString() { final NumberFormat nf = NumberFormat.getInstance(); return new StringBuilder("min = ") .append(nf.format(minimumDuration)) .append(" ns, max = ") .append(nf.format(maximumDuration)) .append(" ns, mean = ") .append(meanDuration) .append(" ns, median = ") .append(medianDuration) .append(" ns, sd = ") .append(standardDeviation) .append(" ns") .toString(); } } 

package io.github.coderodde.statistics.run; import java.util.ArrayList; import java.util.List; import java.util.Objects; /** * This class provides methods for obtaining the running time statistics. * * @author Rodion "rodde" Efremov */ public final class Runner { private static final int MINIMUM_ITERATIONS = 1; public static RunStatistics measure(final Runnable runnable, final int iterations) { Objects.requireNonNull(runnable, "The input runnable is null"); checkIterations(iterations); long minimumDuration = Long.MAX_VALUE; long maximumDuration = Long.MIN_VALUE; long meanDuration = 0; double medianDuration; double standardDeviation; final List<Long> durations = new ArrayList<>(iterations); for (int iteration = 0; iteration < iterations; iteration++) { final long ta = System.nanoTime(); runnable.run(); final long tb = System.nanoTime(); final long duration = tb - ta; minimumDuration = Math.min(minimumDuration, duration); maximumDuration = Math.max(maximumDuration, duration); meanDuration += duration; durations.add(duration); } meanDuration /= iterations; medianDuration = computeMedianDuration(durations); standardDeviation = computeStandardDeviation(durations, meanDuration); return new RunStatistics(minimumDuration, maximumDuration, meanDuration, medianDuration, standardDeviation); } public static RunStatistics measure(final List<Runnable> runnables) { Objects.requireNonNull(runnables, "The input runnables is null"); if (runnables.isEmpty()) { throw new IllegalArgumentException("Nothing to measure"); } long minimumDuration = Long.MAX_VALUE; long maximumDuration = Long.MIN_VALUE; long meanDuration = 0; double medianDuration; double standardDeviation; final List<Long> durations = new ArrayList<>(runnables.size()); for (final Runnable runnable : runnables) { final long ta = System.nanoTime(); runnable.run(); final long tb = System.nanoTime(); final long duration = tb - ta; minimumDuration = Math.min(minimumDuration, duration); maximumDuration = Math.max(maximumDuration, duration); meanDuration += duration; durations.add(duration); } meanDuration /= runnables.size(); medianDuration = computeMedianDuration(durations); standardDeviation = computeStandardDeviation(durations, meanDuration); return new RunStatistics(minimumDuration, maximumDuration, meanDuration, medianDuration, standardDeviation); } private static double computeMedianDuration(final List<Long> durations) { if (durations.size() % 2 == 1) { return durations.get(durations.size() / 2); } else { final int loIndex = durations.size() / 2 - 1; final int hiIndex = durations.size() / 2; return (durations.get(loIndex) + durations.get(hiIndex)) / 2.0; } } private static double computeStandardDeviation(final List<Long> durations, final long meanDuration) { double sum = 0.0; for (final Long duration : durations) { sum += Math.pow(duration - meanDuration, 2.0); } return Math.sqrt(sum / durations.size()); } private static void checkIterations(final int iterations) { if (iterations < MINIMUM_ITERATIONS) { final String exceptionMessage = String.format("Number of iterations (%d) is too small. " + "Must be at least %d.", iterations, MINIMUM_ITERATIONS); throw new IllegalArgumentException(exceptionMessage); } } } 

package io.github.coderodde.statistics.run.demo; import io.github.coderodde.statistics.run.RunStatistics; import io.github.coderodde.statistics.run.Runner; import java.util.Random; public class Demo { private static final Random RANDOM = new Random(13L); public static void main(String[] args) { final RunStatistics runStatistics = Runner.measure(() -> { try { Thread.sleep(RANDOM.nextInt(1_000)); } catch (InterruptedException ex) {} }, 10); System.out.println(runStatistics); } } 

Typical output

min = 101 821 100 ns, max = 998 058 800 ns, mean = 4.9199821E8 ns, median = 5.369435E8 ns, sd = 2.965745976223299E8 ns 

Critique request

Please, tell me anything that comes to mind.

\$\endgroup\$

3 Answers 3

3
\$\begingroup\$

I'm more of a mid-java guy than a JAVA guy, and I am not really sure about possible enhancements, but this is what came to my mind:

  1. Median calculation requires sorting

Right now, computeMedianDuration assumes the list is ordered, but durations are added in run order. Without sorting, I think the median is incorrect.

  1. Mean type mismatch:

You store meanDuration as a long during accumulation, then divide by iterations. This truncates precision. Consider using double throughout.

  1. Standard deviation formula:

I'm not totally sure of this one, but your script currently uses population standard deviation (sqrt(sum/n)), not sample (sqrt(sum/(n-1))). Depending on intent, you may want the sample version.

  1. Iteration validation:

MINIMUM_ITERATIONS = 1 is fine, but in practice, statistical measures are meaningless with 1 run. A higher minimum (e.g., 5 or 10) might be more realistic.

  1. Formatting consistency:

toString() mixes formatted longs with raw doubles. You could format all values consistently with NumberFormat or DecimalFormat.

Overall, I think its a neat, minimal utility for quick runtime checks, but for correctness and reliability, try sorting for median and using double for mean.

New contributor
Chip01 is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
\$\endgroup\$
3
\$\begingroup\$

Computing median

It looks like your computation of median is incorrect. At no point in the following method do you appear to have sorted the data in your list before getting the middle point or mean of the two middle points.

 private static double computeMedianDuration(final List<Long> durations) { if (durations.size() % 2 == 1) { return durations.get(durations.size() / 2); } else { final int loIndex = durations.size() / 2 - 1; final int hiIndex = durations.size() / 2; return (durations.get(loIndex) + durations.get(hiIndex)) / 2.0; } } 

Consider a dataset: {3, 7, 2, 9, 1}. By your logic, the median is 2, but sorted the dataset is {1, 2, 3, 7, 9} and the median is correctly 3.

As a sidenote, it appears you have taken my past comments about statistics to heart. This is great!

Record classes

If you're not concerned with being compatible with older versions of Java, RunStatistics would be an ideal place to use a record class to reduce a lot of boilerplate.

\$\endgroup\$
2
  • \$\begingroup\$ Would this work? private static double computeMedianDuration(final List<Long> durations) { Collections.sort(durations); int size = durations.size(); if (size % 2 == 1) { return durations.get(size / 2); } else { return (durations.get(size / 2 - 1) + durations.get(size / 2)) / 2.0; } } \$\endgroup\$ Commented 15 hours ago
  • \$\begingroup\$ Yes, you just need to sort the durations first. Though this will not change the size, so you could set that variable first. \$\endgroup\$ Commented 15 hours ago
1
\$\begingroup\$

You haven't tagged reinventing-the-wheel.
You do not use java.util.LongSummaryStatistics - no standardDeviation there…

(Computing the median does not require the values to be ordered or put into min&max heaps.)

\$\endgroup\$

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.