[Bug]: Nemotron SuperV3 BF16 crashing during GSM8K accuracy test with attention dp = off
Open
Nobody has claimed this yet.
bug
Inference runtime
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
System Info
See nvbug 5919796
Who can help?
No response
Information
- The official example scripts
- My own modified scripts
Tasks
- An officially supported task in the
examplesfolder (such as GLUE/SQuAD, ...) - My own task or dataset (give details below)
Reproduction
pytest accuracy/test_llm_api_autodeploy.py::TestNemotronSuperV3::test_accuracy[bf16-4-attn_dp_off-trtllm] -s
pytest accuracy/test_llm_api_autodeploy.py::TestNemotronSuperV3::test_accuracy[bf16-4-attn_dp_off-flashinfer] -s
Expected behavior
Test passes
actual behavior
Test crashes
additional notes
N/A
Before submitting a new issue...
- Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the two listed pytest cases in accuracy/test_llm_api_autodeploy.py for Nemotron SuperV3 with BF16 and attention dp off, then inspect the resulting crash and the referenced nvbug 5919796. Done means both TRT-LLM and FlashInfer accuracy tests complete successfully instead of crashing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100