borrowbreadcat / borrowbreadcat/Kinematics-Wizard

floating point rounding issue

Open
#1 0 comments 0 reactions 0 assignees View on GitHub
bug help wanted
Dominant language
C++
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

It is a known, well-documented hurdle for everyone in computer science fields. Floating point variables are calculated in base-2 (binary) but represented in base-10 (decimal). As a simple example, consider the base-ten numbers 0.1 and 0.33333-repeating. 0.1 is a precise representation of the number whereas 0.33333 is only an approximation for the actual number 0.33333-repeating. In base 10, we are not able to precisely calculate with 0.3-repeating without some rounding involved. The base-ten number 0.1 cannot be represented precisely in base-two. Instead it is a repeating fraction which must be rounded for calculations, much the way base-ten calculations handle 0.3-repeating. So when the user tells the program that acceleration is equal to 9.8, that number is converted from decimal to binary (rounded if necessary) before it is used in calculations. While it usually results in minor inaccuracies, when the rounded floating point value is used in division or square root calculations, occasionally the final answer may be very inaccurate.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.