fix(QUANT-GGUF-CPU-THREADPOOL): the CPU thread count reads hardware_concurrency() and ignores the cgroup quota
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: QUANT-GGUF-CPU-THREADPOOL
Found by wave PORTQ-2 re-deriving PORT-NOW entry 66 of 5559679229..e126687a9a
(#2646). Upstream:
7635a9002b vllm#49919, "[Core] Explicitly manage torch CPU threads in workers".
Two of the commit's three halves are surface-absent; one is not
Surface-absent: there is no torch, so there is no intra-op thread pool to drop to
1, and there is no multiprocess executor — the executor is a single-process
pass-through (include/vllm/v1/executor/executor.h:41-47, "upstream
UniProcExecutor collapsed to a direct call"), so multiproc_executor.py and
gpu_worker.py have nothing to edit.
The half that does have a surface is the CPU-count resolution, and it is
quota-blind in exactly the way upstream's available_cpu_count() fixes.
src/vt/cpu/cpu_threadpool.cpp:111-120:
int ThreadsFromEnv() {
int n = 0;
if (const char* e = std::getenv("VLLM_CPP_CPU_THREADS")) {
n = std::atoi(e);
}
if (n <= 0) {
n = static_cast<int>(std::thread::hardware_concurrency());
}
return std::clamp(n, 1, kMaxThreads);
}
std::thread::hardware_concurrency() reports HOST cores inside a cpu-limited
container. Searches returning zero across src/ and include/: cgroup,
cpu.max, cpuset, sched_getaffinity, OMP_NUM_THREADS, set_num_threads.
This is not hypothetical for this project: AGENTS.md's own lease model runs jobs
in containers, and .agents/environment.md records that unconstrained
parallelism has OOM-rebooted the dgx box.
Scope
A cgroup- and affinity-aware usable-CPU count behind ThreadsFromEnv(): read
sched_getaffinity, then the tightest cpu.max / cpu.cfs_quota_us up the
hierarchy, take the minimum, floor at 1. The same treatment is owed to the two
other hardware_concurrency() sites,
src/vllm/model_executor/model_loader/nvfp4_dequant.cpp:144 and
src/vllm/entrypoints/openai/api_server.cpp:73.
Roughly 60-90 lines over 1-3 files, plus a hermetic test over synthetic cgroup
files. The explicit VLLM_CPP_CPU_THREADS override must keep priority.
Honest weight
Nothing was measured. The case is the code plus the container fact in
AGENTS.md, not an observed regression. Whether an over-subscribed pool has cost a
measurement on the leased containers is unknown, and this issue does not claim it
has.
Nothing was executed for this finding: no build, no test, no GPU.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with src/vt/cpu/cpu_threadpool.cpp:111-120 and inspect the two other hardware_concurrency() sites in src/vllm/model_executor/model_loader/nvfp4_dequant.cpp:144 and src/vllm/entrypoints/openai/api_server.cpp:73. Add a hermetic test using synthetic cgroup files and verify affinity and the tightest quota are respected while VLLM_CPP_CPU_THREADS remains the override. Done means all three sites use the usable CPU count without requiring a live container.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100