1
$\begingroup$

I noticed that SetPrecision doesn't actually guarantee that my number is as precise as I want:

prec=20; SetPrecision[1.0, prec] SetPrecision[1.1, prec] Out[666]= 1.0000000000000000000 Out[667]= 1.1000000000000000888

I know one way to prevent this would be to replace 1.1 with 11/10, but this gets pretty cumbersome when adding more digits after the decimal. Another way is to use 1.1`20 (thanks @J.M.) but I need to use a variable precision. Is there a better way?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Try 1.1`20. $\endgroup$ Commented Jun 8, 2016 at 2:44
  • $\begingroup$ @J.M. That works if you're not using a variable for precision (i.e. SetPrecision[1.1, prec]), but in my case I am. Your answer would be fine if 1.1`prec was a thing. See a previous question of mine: mathematica.stackexchange.com/questions/89837/… $\endgroup$ Commented Jun 8, 2016 at 4:19
  • 1
    $\begingroup$ You should have clarified what you wanted then; there is no indication that you want such a thing in your question. In any case: you might want to consider Rationalize[], but that doesn't work in general. $\endgroup$ Commented Jun 8, 2016 at 4:22
  • $\begingroup$ Clarified my question. Rationalize[] actually works great! I'll only be using finite decimal places of course. $\endgroup$ Commented Jun 8, 2016 at 4:34
  • 1
    $\begingroup$ Possibly informative: youtube.com/watch?v=PZRI1IfStY0 $\endgroup$ Commented Jun 8, 2016 at 12:25

1 Answer 1

Reset to default
2
$\begingroup$

While not being a Mathematica expert, I assume that everything works as expected: your 1.1 "actually is" 1.1000000000000000888 since it is probably stored with a binary form in the IEEE754 standard (count the number of exact decimal digits and remember that IEEE754 standard with double-precision type has 15/16 exact decimal digits). See http://mathworld.wolfram.com/Floating-PointArithmetic.html

You can later (even in the same line!) increse the precision as much as you want but your 1.1 already is 1.1000000000000000888 and the displayed result is exactly what it should.

If this is really an issue for you, you should avoid using floating-point numbers (even when typing them in your script) either by using fractions or maybe by using decimal digits in strings and converting them directly from strings (but it is much slower).

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Also look at the first and sixth points under "Details" in the docs for SetPrecision[]. $\endgroup$ Commented Jun 8, 2016 at 11:57

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.