Benchmark: the yang-gb1 benchmark asserts nothing about its result
Nobody has claimed this yet.
- Dominant language
- Macaulay2
- Stars
- 435
- Forks
- 297
- Avg merge
- 4d 20h
- Merged PRs (30d)
- 11
Description
This issue was triaged from [`bugs/dan/0.5-Benchmark-package`](https://github.com/Macaulay2/M2/blob/388c1ff0ce30d83751dea7bc7eac77fdc1305dd7/bugs/dan/0.5-Benchmark-package), one of the 857 files removed from the pre-GitHub `bugs/` tree by [`d2c8d27826`](https://github.com/Macaulay2/M2/commit/d2c8d27826) and catalogued in [#36](https://github.com/Macaulay2/M2/issues/36). **The commentary below was written by Claude (Claude Opus 5, via Claude Code)**, not by @d-torrance, whose account posted it -- please weigh it accordingly.
### The original file, verbatim
```text
Benchmark package:
Example "yang-gb1" has no assertion to test the result.
Tests of the package should include running each the benchmarks.
```
### Where it stands today
Half done, by Dan himself.
**The tests half landed**: `8753fd3ae7` (2010) added `scan(keys benchmarks, b -> TEST("runBenchmarks "|format b))`,
so every benchmark now runs as a test — which is the file's second sentence.
**The assertion half did not.** `yang-gb1` still ends
```m2
(ti,re) := toSequence timing gb(J1); ti)
```
with no assertion about the result. So the benchmark measures how long a Gröbner basis computation takes
without checking that it computed the right thing — which is the file's first sentence, and the more
important half: a benchmark that silently computes the wrong answer looks like an improvement.
### Notes for whoever picks this up
The other benchmarks are worth a glance at the same time, since the file singles out `yang-gb1` only as
the example it noticed. An assertion here can be cheap — the number of generators of the result, or its
Betti numbers — without slowing the timing being measured.
`open` · disposition `issue` · source of truth: [`bug-triage/catalog.tsv`](https://github.com/d-torrance/M2/blob/bug-triage/bug-triage/catalog.tsv)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the `yang-gb1` benchmark and compare its result handling with the other benchmarks, then review the existing `runBenchmarks` test registration. Add a correctness assertion that does not affect the timing measurement, and run the benchmark package tests to confirm the result is checked.
Written by the indexing model from the issue text.
Assessment
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100