[CI Problem] test_solution_consistency fails intentionally-nondeterministic check in solve_linear_equations
- Dominant language
- Python
- Stars
- 13.7k
- Forks
- 4k
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 112
Description
### Branch/PR Failing
https://github.com/apache/tvm/pull/19966
### Jenkins Link
https://ci.tlcpack.ai/blue/organizations/jenkins/tvm-cpu/detail/PR-19966/1/pipeline
### Flakiness
`test_solution_consistency` is explicitly documented in the test file as "intentionally non-deterministic" (it seeds `random` fresh each run and prints the seed only on failure), so it is expected to be flaky by design; the seed for this particular failure wasn't captured in the CI log.
The failing PR only touches `python/tvm/relax/frontend/onnx/onnx_frontend.py` and its test file, with no relation to `arith`, so the failure looks environment/randomness-driven rather than caused by the PR's own changes.
Failure:
```
AssertionError: Expression x1 * 3 - x0 * 5 - 1 != x0 * 3 + 1 - x1 * 3
is not true on {x1: I.Range(-20, 21), x0: I.Range(-20, 21)}
Counterexample: x0 = -13, x1 = -17
```
raised from `_check(num_vars=2, num_formulas=1)` in `tests/python/arith/test_arith_solve_linear_equations.py::test_solution_consistency`.
### Triage
* needs-triage
* flaky
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with tests/python/arith/test_arith_solve_linear_equations.py::test_solution_consistency and its _check(num_vars=2, num_formulas=1) path; run the focused test repeatedly to reproduce the failure and inspect the emitted seed. Compare the documented nondeterminism with the CI failure handling, and confirm the test no longer produces an unaccounted failure or provides enough seed information to diagnose one.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- compilers, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100