-1
$\begingroup$

I was working on an exercise from a chemistry textbook which was about an application of the osmotic pressure formula.
When I checked the solution the answer give was $62.05$ while I was getting as a result $63.4$.
I was going over the logic and I could not find a problem. In the end I found out that the difference is because I rounded a bit.
Specifically: the pressure given was $560$mm and when I converted it to standard atm ($\frac{560}{760}$) I kept as a result $0.73$ atm instead of $0.7368$ atm.
Then when applying the formula $\frac{0.73\times 0.1}{0.082\times 303}$ I kept as a result $0.0029$ instead of $0.0029656$
So these two truncations (multiplying $0.1$ with $0.73$ instead of $0.7368$ and dividing by $0.0029$ instead of $0.0029656$ ended up in the difference in the solution.

So in general, when working on a problem what is the approach/convention on how many decimals or when to truncate when doing calculations to end up with results that match that of others?

$\endgroup$
3
  • $\begingroup$ Comments have been moved to chat; please do not continue the discussion here. Before posting a comment below this one, please review the purposes of comments. Comments that do not request clarification or suggest improvements usually belong as an answer, on Mathematics Meta, or in Mathematics Chat. Comments continuing discussion may be removed. $\endgroup$ Commented Nov 4 at 19:40
  • $\begingroup$ If this should be asked to e.g. engineering SE, can someone help me move it? I think deleting the question causes penalty $\endgroup$ Commented Nov 4 at 19:53
  • $\begingroup$ There is no penalty. $\endgroup$ Commented Nov 4 at 21:14

1 Answer 1

Reset to default
1
$\begingroup$

If you want to obtain an answer with $N$ significant digits, you should keep, throughout your calculations, numbers rounded to $N$ significant digits. The first significant digit starts after the zeros end. For example, 0.0029656 rounded to four significant digits is 0.002966 (and not 0.002965 , because you must apply rounding, not simply truncate digits). 0.7368 already has four significant digits, so it stays as it is. Writing 0.73 instead is incorrect — you already introduce an error in the second significant digit, since the correct rounding to two significant digits is 0.74.

$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.