Macaulay2 / Macaulay2/M2

Arithogeometric mean of infinity inconsistencies

Open
#4,416 0 comments 0 reactions 0 assignees View on GitHub
Interpreter
Dominant language
Macaulay2
Stars
435
Forks
297
Avg merge
4d 20h
Merged PRs (30d)
11

Description

We currently compute the `agm` of two numbers in three distinct ways:

* `RR`: using MPFR
* `CC`: using our own function in `gmp.d` (and I'm proposing we use FLINT in #4347)
* `RRi`/`CCi`: using FLINT

This results in inconsistent output when one of the arguments is infinite:

```m2
i1 : agmTests = f -> {
agm(f numeric infinity, f 0),
agm(f numeric infinity, f 2),
agm(f 0, f numeric infinity),
agm(f 2, f numeric infinity)}


o1 = agmTests

o1 : FunctionClosure

i2 : agmTests toRR

o2 = {NotANumber, infinity, NotANumber, infinity}

o2 : List

i3 : agmTests toCC

o3 = {infinity, infinity, infinity, infinity}

o3 : List

i4 : agmTests toRRi

o4 = {[NotANumber,NotANumber] (an empty interval), [NotANumber,NotANumber] (an empty interval), [NotANumber,NotANumber] (an empty interval), [NotANumber,NotANumber] (an empty interval)}

o4 : List

i5 : agmTests toCCi

o5 = {[NotANumber,NotANumber]+[NotANumber,NotANumber]*ii (an empty interval), [NotANumber,NotANumber]+[NotANumber,NotANumber]*ii (an empty interval), [NotANumber,NotANumber]+[NotANumber,NotANumber]*ii (an empty interval),
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[NotANumber,NotANumber]+[NotANumber,NotANumber]*ii (an empty interval)}

o5 : List
```

Maybe it makes sense for different behavior between the real and complex cases, but certainly it seems like we'd want `RR` and `RRi` to agree and to `CC` and `CCi` to agree.

I asked Claude to classify what it thinks the output should be:

| Case | `agm(∞, 2)` | `agm(∞, 0)` |
|---|---|---|
| **Real** | `+∞` | `NaN` |
| **Complex** | `NaN` | `NaN` |

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the RR and CC implementations, including the own function in gmp.d, then compare the RRi/CCi FLINT paths and the proposed direction in #4347. Done means the real and interval results agree, and the real and complex cases consistently follow the expected infinity and NaN behavior shown in the issue.

Written by the indexing model from the issue text.

Assessment

Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.