NVIDIA-NeMo / NVIDIA-NeMo/RL

[TRAIN-9] Thinking models (Qwen3/Qwen3.5) silently produce 0% accuracy due to low max_new_tokens

Open
#2,725 2 comments 0 reactions 1 assignee Claimed by @sharonyu-115 View on GitHub
accuracy bug Documentation
Dominant language
Python
Stars
2k
Forks
561
Avg merge
4d 5h
Merged PRs (30d)
145

Description

## Problem

Default `max_new_tokens` values in Qwen2.5 and Llama-3 GRPO recipes (3K-4K tokens) are too low for thinking models like Qwen3 and Qwen3.5. Reasoning traces exceed the limit and get truncated before the final `\boxed{N}` answer. Evaluation accuracy appears near 0% even though training loss and reward metrics look normal. There is no documentation warning about this when switching to thinking models.

## Steps to reproduce

1. Switch a GRPO config from a non-thinking baseline to Qwen3 or Qwen3.5.
2. Run training and observe evaluation accuracy near 0%.
3. Increase `max_new_tokens` in the vLLM config to >= 8K and confirm accuracy recovers.

## Suggested fix

- Add a documentation note in the Qwen recipe guides to set `max_new_tokens` to at least 8K for thinking models.
- Consider adding a config validation warning when a thinking model is detected but `max_new_tokens` is below a minimum threshold.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.