1

This part of code should speak for itself, my question is can I get 1242.08 result in res variable, and not 1242.07, because in math

b = 1/(1/b)

and in double which has less precision it seems I got good result

Can I fix decimal part of calculation to give me mathematically right result:

decimal rate = 124.2075M; decimal amount = 10M; decimal control = decimal.Round(rate * amount, 2); //it is 1242.08 //but decimal res = decimal.Round(amount * (1 / (1 / rate)), 2);//1242.07 - should be 1242.08 decimal res1 = decimal.Round(amount * 124.2075M, 2);//1242.08 OK ///////////////////////////////////////////////////////////// //while with double seems OK double ratef = 124.2075; double amountf = 10; double resf = Math.Round(amountf * (1 / (1 / ratef)), 2);//1242.08 OK double res1f = Math.Round(amountf * 124.2075, 2);//1242.08 OK
Improve this question
5
  • You can try Math.Round(1242.075M*100)/100. Commented Sep 8, 2016 at 13:51
  • The identity b = 1/(1/b) which is good in pure math (for non-zero b), simply does not hold for floating-point arithmetic. In particular, when rate is 124.2075M, then 1M / (1M / rate) is something else, namely 124.20749999999999999999999991M. The good thing about decimal is that you can clearly see from the printed representation of the value, that it is not the same. (For double and float, sometimes there is "hidden" precision unless you use a special format string for printing the value.) Commented Sep 8, 2016 at 14:10
  • Just do this yourself with a piece of paper and use long division to compute 1 / 124.2075 as you learned in elementary school. The paper is not big enough to write the infinite sequence of digits in the result, you have to stop somewhere. And if you do it again with the truncated result you wrote down then you of course don't get the original value back. Computers don't work any differently, they also don't have infinite storage. Your assumption that this should be possible is simply not correct. Commented Sep 8, 2016 at 15:19
  • If you want pretty numbers then multiply everything by a 10000 and use integer math. It is only prettier, it is not more correct. Commented Sep 8, 2016 at 15:22
  • @Hans Passant - it was suprising to me that an erorr on 29th decimal, can lead to an error on the second decimal in the next step of calculation :) Commented Sep 9, 2016 at 6:50

1 Answer 1

Reset to default
5

That's a limitation of the datatype decimal that can hold up to 29 digits

The result of the first calculation (res1) does not fit into decimal so you get a invalid/imprecisely result.

decimal rate = 124.2075M; decimal amount = 10M; decimal res1 = (1 / rate); //0.0080510436165287925447336111M <- not enough decimal places decimal res2 = (1 / res1); //124.20749999999999999999999991M decimal res3 = amount * res2; //1242.0749999999999999999999991M decimal res4 = decimal.Round(res3, 2); //1242.07M <- correct rounding
Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

I have noticed that if I round 1/rate on something like 10 digits I got 1242.08, is this by accident? Or can I use something like that to fix this?
That's by accident because you just cut off everything after 0.0080510436. But that's not a general solution for all numbers - that just works for this one
Should I use different data type for this calculation, I saw that with Double I get right result, is it also by accident that double seems to work?
@VladimirRadojicic That is not a complete accident. With decimal, the very midpoint where rounding has two equally close "destinations" to choose between, is actually exactly representable. For example 1242.075 (the midpoint between 1242.07 and 1242.08) is exactly representable as a decimal, but not as a double. I guess that makes a difference because you round based on decimally "nice" numbers. However, in principle, the same errors occur with binary types like double. Edit: A double example: Math.Round(1452.375, 2) != Math.Round(1.0 / (1.0 / 1452.375))
(I meant , 2 before the final ) in above comment, of course.)
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.