[Bug]: Sporadic hangs in NanoV3 accuracy tests
Open
Nobody has claimed this yet.
bug
Inference runtime
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
System Info
See nvbug 5921674
Who can help?
No response
Information
- The official example scripts
- My own modified scripts
Tasks
- An officially supported task in the
examplesfolder (such as GLUE/SQuAD, ...) - My own task or dataset (give details below)
Reproduction
pytest accuracy/test_llm_api_autodeploy.py::TestNemotronNanoV3::test_accuracy[fp8-1-trtllm]
pytest accuracy/test_llm_api_autodeploy.py::TestNemotronNanoV3::test_accuracy[fp8-4-trtllm]
Expected behavior
Test passes consistently
actual behavior
Test is sporadically hanging. Perhaps when running multiple test cases
additional notes
N/A
Before submitting a new issue...
- Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the two named pytest cases in accuracy/test_llm_api_autodeploy.py: TestNemotronNanoV3::test_accuracy[fp8-1-trtllm] and [fp8-4-trtllm]. Compare their behavior across repeated and combined runs to isolate the sporadic hang. Done means both accuracy tests complete and pass consistently.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100