ROCm / ROCm/ATOM

CI: Qwen3-Next-80B-A3B-Thinking accuracy test times out at 30-minute limit

Open
#565 0 comments 0 reactions 2 assignees View on GitHub

@valarLip is already working on this.

Since Apr 14, 2026.

Dominant language
Python
Stars
184
Forks
149
Avg merge
2d 7h
Merged PRs (30d)
189

Description

Problem

The Qwen3-Next-80B-A3B-Thinking accuracy test consistently fails with a 30-minute timeout in ATOM Test CI. This is the sole cause of CI failure in the latest run (#24389078026) where 12/13 accuracy jobs passed.

##[error]The action 'Run ATOM accuracy test' has timed out after 30 minutes.

Impact

  • ATOM Test on main has not had a fully green run since 2026-04-04 (commit 42c0449aa2)
  • Every run since then fails due to this timeout (and occasional infra flakes)
  • This blocks release readiness assessment — the CI signal is unreliable

Evidence: Not an Accuracy Regression

Dashboard history shows Qwen3-Next-80B-A3B-Thinking accuracy is stable at 0.68–0.71 (GSM8K) across all runs from 4/3 to 4/9:

Date Commit Score
4/03 8d04949f 0.6998
4/04 42c0449a 0.6990
4/05 be228160 0.6808
4/06 444a8d43 0.6929
4/07 749aa057 0.6975
4/08 7618e5a5 0.7119
4/09 085276c5 0.6907

The model produces results when given enough time. The 30-minute timeout is simply too short.

Suggested Fix

Increase the timeout for this specific model's accuracy job (e.g., to 60 minutes), or split the model download and test into separate steps so download time doesn't count against the test timeout.

Related

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.