Macaulay2 / Macaulay2/M2

Benchmark: the yang-gb1 benchmark asserts nothing about its result

Open Beginner friendly
#4,541 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bugs directory good beginners' project
Dominant language
Macaulay2
Stars
435
Forks
297
Avg merge
4d 20h
Merged PRs (30d)
11

Description

This issue was triaged from [`bugs/dan/0.5-Benchmark-package`](https://github.com/Macaulay2/M2/blob/388c1ff0ce30d83751dea7bc7eac77fdc1305dd7/bugs/dan/0.5-Benchmark-package), one of the 857 files removed from the pre-GitHub `bugs/` tree by [`d2c8d27826`](https://github.com/Macaulay2/M2/commit/d2c8d27826) and catalogued in [#36](https://github.com/Macaulay2/M2/issues/36). **The commentary below was written by Claude (Claude Opus 5, via Claude Code)**, not by @d-torrance, whose account posted it -- please weigh it accordingly.

### The original file, verbatim

```text
Benchmark package:

Example "yang-gb1" has no assertion to test the result.

Tests of the package should include running each the benchmarks.
```

### Where it stands today

Half done, by Dan himself.

**The tests half landed**: `8753fd3ae7` (2010) added `scan(keys benchmarks, b -> TEST("runBenchmarks "|format b))`,
so every benchmark now runs as a test — which is the file's second sentence.

**The assertion half did not.** `yang-gb1` still ends

```m2
(ti,re) := toSequence timing gb(J1); ti)
```

with no assertion about the result. So the benchmark measures how long a Gröbner basis computation takes
without checking that it computed the right thing — which is the file's first sentence, and the more
important half: a benchmark that silently computes the wrong answer looks like an improvement.

### Notes for whoever picks this up

The other benchmarks are worth a glance at the same time, since the file singles out `yang-gb1` only as
the example it noticed. An assertion here can be cheap — the number of generators of the result, or its
Betti numbers — without slowing the timing being measured.

`open` · disposition `issue` · source of truth: [`bug-triage/catalog.tsv`](https://github.com/d-torrance/M2/blob/bug-triage/bug-triage/catalog.tsv)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the `yang-gb1` benchmark and compare its result handling with the other benchmarks, then review the existing `runBenchmarks` test registration. Add a correctness assertion that does not affect the timing measurement, and run the benchmark package tests to confirm the result is checked.

Written by the indexing model from the issue text.

Assessment

Domain
testing-qa
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.