fix: Pass NVML CC settings by reference in release/1.2.1
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
TensorRT-LLM 1.2.1 can segfault during worker initialization with nvidia-ml-py==13.610.43. On H200 NVL with driver 610.57.04, the failure occurs after model weights load and before the HTTP server becomes ready, in both GPU CC-off and TDX/GPU CC-on deployments.
The release passes c_nvmlSystemConfComputeSettings_v1_t by value to nvmlSystemGetConfComputeSettings. The pynvml wrapper forwards the argument to the native NVML function, which expects a pointer. Passing ctypes.byref(cc_settings) resolves the argument mismatch.
The pointer-based call is already present on main through #16850. The proposed release/1.2.1 backport applies this correction while preserving the release's existing CC/NVLE classification. All four added regression cases fail against the original source and pass with the correction.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Compare the release/1.2.1 NVML call with the pointer-based call already present on main through #16850, focusing on pynvml's handling of c_nvmlSystemConfComputeSettings_v1_t. Apply the backport while preserving the release's existing CC/NVLE classification, then run the four added regression cases and confirm they fail before and pass after the correction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100