borrowbreadcat / borrowbreadcat/Kinematics-Wizard
floating point rounding issue
- Dominant language
- C++
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
It is a known, well-documented hurdle for everyone in computer science fields. Floating point variables are calculated in base-2 (binary) but represented in base-10 (decimal). As a simple example, consider the base-ten numbers 0.1 and 0.33333-repeating. 0.1 is a precise representation of the number whereas 0.33333 is only an approximation for the actual number 0.33333-repeating. In base 10, we are not able to precisely calculate with 0.3-repeating without some rounding involved. The base-ten number 0.1 cannot be represented precisely in base-two. Instead it is a repeating fraction which must be rounded for calculations, much the way base-ten calculations handle 0.3-repeating. So when the user tells the program that acceleration is equal to 9.8, that number is converted from decimal to binary (rounded if necessary) before it is used in calculations. While it usually results in minor inaccuracies, when the rounded floating point value is used in division or square root calculations, occasionally the final answer may be very inaccurate.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.