19

I need to find the time elapsed between two functions doing the same operation but written in different algorithm. I need to find the fastest among the two

Here is my code snippet

Stopwatch sw = new Stopwatch(); sw.Start(); Console.WriteLine(sample.palindrome()); // algorithm 1 sw.Stop(); Console.WriteLine(sw.ElapsedMilliseconds);//tried sw.elapsed and sw.elapsedticks sw.Reset(); //tried with and without reset sw.Start(); Console.WriteLine(sample.isPalindrome()); //algorithm 2 sw.Stop(); Console.WriteLine(sw.ElapsedMilliseconds);

Technically this should give the time taken for two algorithms. This gives that the algorithm 2 is faster. But it gives different time if I interchange the calling of two function. Like if I call algorithm2 first and algorithm1 second it says algorithm1 is faster.

I dont know what I am doing wrong.

Improve this question
2
  • 7
    C# Performance Benchmark Mistakes, Part One, by Eric Lippert. Specifically, yours is similar to the one at part three: Mistake #6: Treat the first run as nothing special when measuring average performance. Commented Dec 15, 2014 at 6:53
  • Yeap, usually the biggest performance benefit you can get is not by searching for the fastest algorithm, but by taking the cache seriously Commented Dec 15, 2014 at 7:50

3 Answers 3

Reset to default
21

I assume your palindrome methods runs extremely fast in this example and therefore in order to get a real result you will need to run them a couple of times and then decide which is faster.
Something like this:

int numberOfIterations = 1000; // you decide on a reasonable threshold. sample.palindrome(); // Call this the first time and avoid measuring the JIT compile time Stopwatch sw = new Stopwatch(); sw.Start(); for(int i = 0 ; i < numberOfIterations ; i++) { sample.palindrome(); // why console write? } sw.Stop(); Console.WriteLine(sw.ElapsedMilliseconds); // or sw.ElapsedMilliseconds/numberOfIterations

Now do the same for the second method and you will get more realistic results.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

With your code you would be measuring the overhead of the method being JIT'd; you sould be calling sampe.palindrome() once before actually measuring anything.
@InBetween - You're correct, but with a threshold of a 1000+ calls you wont really notice it in 99.99999999999999% of the cases. Anyway I'll update my answer to be a bit more precise. Thanks.
I agree, but its still worth pointing it out as it is a common mistake when measuring performance.
8

What you must do is execute both methods before the actual calculated tests for the compiled code to be JIT'd. Then test with multiple tries. Here is a code mockup.

The compiled code in CIL format will be JIT'd upon first execution, it will be translated into machine code. So testing it at first is in-accurate. So let the code be JIT'd before actually testing it.

sample.palindrome(); sample.isPalindrome(); Stopwatch sw = Stopwatch.StartNew(); for (int i = 0; i < 1000; i++) { sample.palindrome(); Console.WriteLine("palindrome test #{0} result: {1}", i, sw.ElapsedMilliseconds); } sw.Stop(); Console.WriteLine("palindrome test Final result: {0}", sw.ElapsedMilliseconds); sw.Restart(); for (int i = 0; i < 1000; i++) { sample.isPalindrome(); Console.WriteLine("isPalindrome test #{0} result: {1}", i, sw.ElapsedMilliseconds); } sw.Stop(); Console.WriteLine("isPalindrome test Final result: {0}", sw.ElapsedMilliseconds);

Read more about CIL and JIT

Improve this answer

1 Comment

One thing I noticed being terribly wrong with this answer is using Console.WriteLine in the code that should be tested for performance. Never do this, as outputting text to console gives you random times and slow down the execution.
4

Unless you provide the code of palindrome and isPalindrome function along with the sample class, I can't do much except speculate.

The most likely reason which I guess for this is that both your functions use the same class variables and other data. So when you call the function for the first time, it has to allocate memory to the variables, whereas the next time you call some other function, those one time expenses have already occurred. If not variables, it could be some other matter, but along the same lines.

I suggest that you call both the functions twice, and note the duration only the second time a function is called, so that any resources which they need to use may have been allocated once, and there's lesser probability of something behind the scenes messing with the result.

Let me know if this works. This is mere speculation on my part, and I may be wrong.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.