[Bug]: Fix TPS/User glitches in BTK
Open
@marinayanov is already working on this.
Since Mar 5, 2026.
bug
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
System Info
spikes of TPS/User is seen in the IBP graphs, especially for VLLM as seen in the image below. This is due to cpu bottleneck in the sampler, and can be fixed by increasing the number of api workers. See this: https://nvidia.slack.com/archives/C09H79C4MB8/p1772128612444079?thread_ts=1772088513.942399&cid=C09H79C4MB8
This should be fixed in all our supported frameworks.
Who can help?
No response
Information
- The official example scripts
- My own modified scripts
Tasks
- An officially supported task in the
examplesfolder (such as GLUE/SQuAD, ...) - My own task or dataset (give details below)
Reproduction
na
Expected behavior
na
actual behavior
na
additional notes
na
Before submitting a new issue...
- Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.