dotnet / dotnet/vblang

[Spec Bug] How to get deterministic floating-point precision

Open
#181 4 comments 1 reaction 0 assignees View on GitHub
Spec-Defect
Dominant language
No language data
Stars
328
Forks
71
PR merge metrics
No merged PRs in 30d

Description

The C# specification provides a way for programmers to get deterministic precision for floating-point operations:

> ### 9.3.7 Floating-point types
>
> Floating-point operations may be performed with higher precision than the result type of the operation. [Example: Some hardware architectures support an “extended” or “long double” floating-point type with greater range and precision than the double type, and implicitly perform all floating-point operations using this higher precision type. Only at excessive cost in performance can such hardware architectures be made to perform floating-point operations with less precision, and rather than require an implementation to forfeit both performance and precision, C# allows a higher precision type to be used for all floating-point operations. Other than delivering more precise results, this rarely has any measurable effects. However, in expressions of the form x * y / z, where the multiplication produces a result that is outside the double range, but the subsequent division brings the temporary result back into the double range, the fact that the expression is evaluated in a higher range format can cause a finite result to be produced instead of an infinity. **To force a value of a floating-point type to the exact precision of its type, an explicit cast can be used**. end example]
>

> ### 11.2.2 Identity Conversion
>
> In most cases, an identity conversion has no effect at runtime. However, since floating point operations may be performed at higher precision than prescribed by their type (§9.3.7), assignment of their results may result in a loss of precision, and **explicit casts are guaranteed to reduce precision to what is prescribed by the type**.

However, the VB specification is less helpful to the programmer

> ### 11.13 Arithmetic Operators
>
> The *, /, \, ^, Mod, +, and – operators are the arithmetic operators.
>
> Floating-point arithmetic operations may be performed with higher precision than the result type of the operation. For example, some hardware architectures support an "extended" or "long double" floating-point type with greater range and precision than the Double type, and implicitly perform all floating-point operations using this higher-precision type. Hardware architectures can be made to perform floating-point operations with less precision only at excessive cost in performance; rather than require an implementation to forfeit both performance and precision, Visual Basic allows the higher-precision type to be used for all floating-point operations. Other than delivering more precise results, this rarely has any measurable effects. However, in expressions of the form x * y / z, where the multiplication produces a result that is outside the Double range, but the subsequent division brings the temporary result back into the Double range, the fact that the expression is evaluated in a higher-range format may cause a finite result to be produced instead of infinity.

Also, the VB spec does not describe the meaning of the `CDbl` operator at all, other than to say "Specific cast keywords coerce expressions into the primitive types". The syntax appears in "11.11 Cast Expressions", but there is otherwise no description.

So, by specification, there is no way to get the benefit that C# programmers have of being able to force deterministic precision in floating-point code. But the compiler can and should do better than is required by the specification. Pre-Roslyn VB compilers did retain a cast that appeared in source (thus forcing the CLR to truncate precision), and Roslyn's VB compiler is being fixed in 15.5 to do so as well. See https://github.com/dotnet/roslyn/issues/22533 and https://github.com/dotnet/roslyn/pull/22552 .

We should put that in the specification when we describe the semantics of `CDbl`.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.