Timeline for Converting a numerical string to the equivalent multipled by 100
Current License: CC BY-SA 3.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Mar 21, 2014 at 12:33 | comment | added | Craig Tullis | Also, my comment that "Math results from decimal types will be as precise as the inputs" was on the simplistic side, but these comment fields are short. The results will be as precise as the least precise of the inputs, and still not always perfect mathematically, and quite slow because the operations are 100% software with no help from specialized hardware. The decimal data type is fundamentally the same thing whether you're dealing with Java, C#, C++, MySQL or SQL Server. This post is informative | |
| Mar 21, 2014 at 12:23 | comment | added | Craig Tullis | @palacsint already answered this, but in a nutshell the issue is simply that floating point data types are imprecise and math operations on floats produce rounding errors at the edge of the precision of the data type (turning 0.0071 into 0.007100000000000001 for instance). That is totally fine as long as the error falls outside the precision you require. Decimal data types (scaled integers) on the other hand are not subject to those rounding errors. Math results from decimal types will be as precise as the inputs. Caveat: decimal operations are extremely slow compared to floats. | |
| Mar 20, 2014 at 15:26 | vote | accept | barak manos | ||
| Mar 20, 2014 at 14:08 | review | Suggested edits | |||
| Mar 20, 2014 at 14:13 | |||||
| Mar 20, 2014 at 13:58 | history | tweeted | twitter.com/#!/StackCodeReview/status/446647069709987840 | ||
| Mar 20, 2014 at 13:11 | answer | added | RoToRa | timeline score: 6 | |
| Mar 20, 2014 at 12:29 | answer | added | palacsint | timeline score: 11 | |
| Mar 20, 2014 at 12:26 | history | edited | barak manos | CC BY-SA 3.0 | added 8 characters in body |
| Mar 20, 2014 at 12:16 | history | asked | barak manos | CC BY-SA 3.0 |