Nightly test failing training time metric check llm_grpo_llama3_2_1b_instruct_1n4g_megatron
Open
bug
Speed
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
**Describe the bug**
The test llm_grpo_llama3_2_1b_instruct_1n4g_megatron has been failing the training time metric check closely multiple times.
**Steps/Code to reproduce bug**
Test failing off main branch
**Additional context**
```
│ FAIL │ mean(data["timing/tr… │ 14.051883888244628 │ mean(data["timing/tra… │
│ │ -6, -1) < 13.4 │ │ -6, -1) < 13.4 │
│ │ │ │ (condition evaluated │
│ │ │ │ to False) │
```
Contributor guide
Assessment
This issue has not been assessed yet.