fslaborg / fslaborg/FSharp.Stats

[Discussion] Consider using checked operators or use consistent overflow prevention

Open
#214 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
F#
Stars
227
Forks
58
Avg merge
55m
Merged PRs (30d)
1

Description

the built in arithmetic operators are unchecked, meaning they can produce overflows (numbers larger/smaller than the max/min value that fits into the target data structure)

Example:

178! is

6235135397241908741680674639927586558582878361231153877729215864681715113389074517106770270394081618113953882713761626619212667044889245663364667862568799843977208366195524719896472807475423880975816549024219691598995799655217698833954618998814300242650289839922492033762220111298560000000000000000000000000000000000000000000

this does not fit into float, and the result of `SpecialFunctions.Factorial.factorial` is `infinity`. This can produce unexpected results in functions that use this result for further arithmetic.

I would suggest to either use checked arithmetic operators, which throw an error when an overflow happens (https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/symbol-and-operator-reference/arithmetic-operators#summary-of-unary-arithmetic-operators), or discuss another appropriate way of handling these issues.

The fact that this was not really encountered or reported before leads me to the assumption that it should be safe to use checked operators.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.