Arithogeometric mean of infinity inconsistencies
- Dominant language
- Macaulay2
- Stars
- 435
- Forks
- 297
- Avg merge
- 4d 20h
- Merged PRs (30d)
- 11
Description
We currently compute the `agm` of two numbers in three distinct ways:
* `RR`: using MPFR
* `CC`: using our own function in `gmp.d` (and I'm proposing we use FLINT in #4347)
* `RRi`/`CCi`: using FLINT
This results in inconsistent output when one of the arguments is infinite:
```m2
i1 : agmTests = f -> {
agm(f numeric infinity, f 0),
agm(f numeric infinity, f 2),
agm(f 0, f numeric infinity),
agm(f 2, f numeric infinity)}
o1 = agmTests
o1 : FunctionClosure
i2 : agmTests toRR
o2 = {NotANumber, infinity, NotANumber, infinity}
o2 : List
i3 : agmTests toCC
o3 = {infinity, infinity, infinity, infinity}
o3 : List
i4 : agmTests toRRi
o4 = {[NotANumber,NotANumber] (an empty interval), [NotANumber,NotANumber] (an empty interval), [NotANumber,NotANumber] (an empty interval), [NotANumber,NotANumber] (an empty interval)}
o4 : List
i5 : agmTests toCCi
o5 = {[NotANumber,NotANumber]+[NotANumber,NotANumber]*ii (an empty interval), [NotANumber,NotANumber]+[NotANumber,NotANumber]*ii (an empty interval), [NotANumber,NotANumber]+[NotANumber,NotANumber]*ii (an empty interval),
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[NotANumber,NotANumber]+[NotANumber,NotANumber]*ii (an empty interval)}
o5 : List
```
Maybe it makes sense for different behavior between the real and complex cases, but certainly it seems like we'd want `RR` and `RRi` to agree and to `CC` and `CCi` to agree.
I asked Claude to classify what it thinks the output should be:
| Case | `agm(∞, 2)` | `agm(∞, 0)` |
|---|---|---|
| **Real** | `+∞` | `NaN` |
| **Complex** | `NaN` | `NaN` |
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the RR and CC implementations, including the own function in gmp.d, then compare the RRi/CCi FLINT paths and the proposed direction in #4347. Done means the real and interval results agree, and the real and complex cases consistently follow the expected infinity and NaN behavior shown in the issue.
Written by the indexing model from the issue text.
Assessment
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100