CI: Qwen3-Next-80B-A3B-Thinking accuracy test times out at 30-minute limit
@valarLip is already working on this.
Since Apr 14, 2026.
- Dominant language
- Python
- Stars
- 184
- Forks
- 149
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 189
Description
Problem
The Qwen3-Next-80B-A3B-Thinking accuracy test consistently fails with a 30-minute timeout in ATOM Test CI. This is the sole cause of CI failure in the latest run (#24389078026) where 12/13 accuracy jobs passed.
##[error]The action 'Run ATOM accuracy test' has timed out after 30 minutes.
Impact
- ATOM Test on main has not had a fully green run since 2026-04-04 (commit
42c0449aa2) - Every run since then fails due to this timeout (and occasional infra flakes)
- This blocks release readiness assessment — the CI signal is unreliable
Evidence: Not an Accuracy Regression
Dashboard history shows Qwen3-Next-80B-A3B-Thinking accuracy is stable at 0.68–0.71 (GSM8K) across all runs from 4/3 to 4/9:
| Date | Commit | Score |
|---|---|---|
| 4/03 | 8d04949f |
0.6998 |
| 4/04 | 42c0449a |
0.6990 |
| 4/05 | be228160 |
0.6808 |
| 4/06 | 444a8d43 |
0.6929 |
| 4/07 | 749aa057 |
0.6975 |
| 4/08 | 7618e5a5 |
0.7119 |
| 4/09 | 085276c5 |
0.6907 |
The model produces results when given enough time. The 30-minute timeout is simply too short.
Suggested Fix
Increase the timeout for this specific model's accuracy job (e.g., to 60 minutes), or split the model download and test into separate steps so download time doesn't count against the test timeout.
Related
- Dashboard: https://rocm.github.io/ATOM/benchmark-dashboard/
- Latest run (1/13 fail): https://github.com/ROCm/ATOM/actions/runs/24389078026
- Last fully green run: https://github.com/ROCm/ATOM/actions/runs/23982615516 (2026-04-04)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.