dotnet / dotnet/machinelearning

Change baseline tests to use the new NumberParseOption.UseSingle

Open
#3,589 1 comment 0 reactions 0 assignees View on GitHub
area-Infrastructure Priority:2 test-enhancement
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

A lot of our tests use baseline files that have serialized numbers in them. We've had a lot of issues in the past comparing these baselines files because of minor changes in how floating point values are written to the output file (sometimes changes in which .NET we are running on, or Windows vs. Linux, etc. cause the values to slightly change).

One cause of these issues is because we are writing `float` (32-bit floating points) to the file, but when we are parsing the values, we are using `double` (64-bit floating points).

At times, we've decreased the `digitsOfPrecision` low enough to tolerate these differences. However, there are cases where `digitsOfPrecision` isn't enough, specifically when we have large values that differ by a digit in the exponential form, for example:

```
3.40282347E+38
```
vs
```
3.4028235E+38
```

To solve this issue, I've added a new option - to parse the numbers using `float.Parse` instead of `double.Parse`.

See the solution added in https://github.com/dotnet/machinelearning/pull/3532.

We should go through the tests where we use a lowered `digitsOfPrecision`, and see if using `float.Parse` fixes the test on all platforms. This may allow us to remove the `digitsOfPrecision` parameter altogether if all these places can be converted to `UseSingle`.

cc @tannergooding

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.