ByteDance-Seed / ByteDance-Seed/Bagel
Bagel using a lot of CPU on my 5090 system
- Dominant language
- Python
- Stars
- 6.2k
- Forks
- 545
- PR merge metrics
- No merged PRs in 30d
Description
It wasn't easy but I was able tu run bagel on my windows+5090 machine. I had to use an unofficial flash-attn, but it worked fine.
Now, the issue is GPU/CPU usage. As you see from the attached image, it clearly uses GPU (vRAM is over 25GB, sometimes it goes up to 29) but:
1) CPU is used at nearly 100%
2) GPU has this weird spiky behaviour and is, on average, less than 10% loaded
my idea is that part of the model is offloaded to CPU so the GPU has to wait for it. But why's that? In the videos I saw on more capable machines (eg. h100) this isn't happening.
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files, tests, or entry points. Start by reproducing Bagel on Windows with an RTX 5090 and the unofficial flash-attn build, then inspect CPU and GPU utilization during inference; done should include a supported explanation for the near-100% CPU use and spiky, low-average GPU load.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100