MegEngine / MegEngine/InferLLM
Thread wakening may be bottom neck for large core systems
- Dominant language
- C++
- Stars
- 752
- Forks
- 94
- PR merge metrics
- No merged PRs in 30d
Description
Result: boost from `3.6~8token/s` to `4.2~4.3token/s` on SG2042.
Analyze: I added tracy to trace execution in details. I observed that worker wakening is problematic. **There are some workers picked up task after the completion of other workers.**
That means the execution time is as twice as the expected sometimes. I guess that it is caused by `thread::yield()`, which switched out the busy waiting loop.
I am thinking about to make a higher/busier performance poll, while being able to switch to an idle state when waiting for user input.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the Tracy traces described in the issue and investigate the worker-wakening path around thread::yield(). Reproduce the delayed task pickup on SG2042, then measure whether a busier polling strategy improves throughput without preventing the workers from becoming idle while waiting for user input.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100