Enable `CpusetMems` when CPU allocation is NUMA-local
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 358
Description
Parent epic: #11216
## Main idea
`src/ai/backend/agent/docker/intrinsic.py:429–442` — the `CpusetMems` host-config key is commented out, so container memory is not pinned to the same NUMA node as the allocated CPU cores. NUMA tracking itself is fully implemented: `AffinityMap` (`src/ai/backend/agent/affinity_map.py:145–160`) already computes the distance matrix, and `CPUPlugin.list_devices()` (`intrinsic.py:213`) records a `numa_node` per core.
Gate `CpusetMems` on the allocated cores belonging to a single NUMA node, and set it to that node ID. Fall back to the current behavior (omit the key) when the allocation spans multiple nodes.
```python
# intrinsic.py, around line 434
numa_nodes = {core.numa_node for core in allocated_cores}
host_config = {"Cpus": len(cores), "CpusetCpus": ",".join(sorted_core_ids)}
if len(numa_nodes) == 1:
host_config["CpusetMems"] = str(next(iter(numa_nodes)))
```
Expected 2–3× improvement on remote-memory access latency for NUMA hosts when the allocation already happens to be node-local (which `AllocationStrategy.FILL` prefers today — see comment at `resources.py:176`).
## Alternative ideas
- Allow multi-node pinning (`CpusetMems="0,1"`) when the allocation spans exactly the selected set. Lower win, wider applicability.
- Always emit `CpusetMems` equal to the union of all nodes covered by allocated cores — functionally equivalent to the current default on most systems, so low value.
## Anything else?
Needs verification on a NUMA host. Single-socket machines are unaffected (single node = current behavior).
JIRA Issue: BA-5857
Contributor guide
Assessment
This issue has not been assessed yet.