gpustack / gpustack/gpustack-operator
todo: verify the new T-Head PPU partition reader on hardware
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 4
- Forks
- 7
- Avg merge
- 3h 9m
- Merged PRs (30d)
- 213
Description
What was left undone
Running the T-Head PPU partition reader against a driver. It is entirely new in #92 — before it, a
partition-backed Instance on a PPU fell through to the per-process pass and reported the parent card,
i.e. every tenant's memory under one tenant's name. The reader has never executed on hardware.
Two things need reading:
- A PPU partition Instance reporting its own identifier, capacity and usage on all three surfaces,
the way NVIDIA MIG now does. - The deliberate limitation: an allocation carrying no recorded identifier is answered as an absence
with a reason. T-Head has no derivation fallback on purpose — HGML enumerates 85 GPU-instance profile
ids where NVML enumerates 17, so translating a recorded profile name back into a profile id would
cost hundreds of driver calls per card per monitor period.
Where it came from
PR #92, pkg/devicemanager/detector/thead/mig_process.go (new file), commit
feat(devicemanager): report a hardware partition on its own handle. Covered by 6 table cases plus a
driver-call-count pin, and by nothing else.
What it needs in order to run
A T-Head PPU host with MIG mode available, reachable as a Kubernetes node. Note the two known
constraints from earlier PPU work: toggling -mig 1 fails EBUSY while any process holds the
driver on any card, and a PPU slice pod needs no runtimeClassName (unlike NVIDIA/MThreads).
How to verify it
- Enable MIG on one PPU, restart the device-manager DaemonSet (a mode toggle fires no re-detect), and
wait for thealibabacloud.com/ppu.partitioned.mig-*node keys. - Create a partition Instance and read the subresource, the exporter and
/monitor/snapshot. Assert:
idis the partition's own UUID and not the parent card's;memoryTotalMiBis the partition's
own capacity read off its handle;coresUtilizationPercentis absent withcoresReason: unsupported. - Confirm the parent card's figures appear nowhere in that entry except temperature, power and
health — which every mode takes from the card by design. cases/case-39.shfor the surrounding PPU claims (pinned placement, a slice capped inside the
container), which also has not run since this branch.- Optionally confirm the no-identifier path by replaying a Pod annotated before the field: the entry
must report an absence with a reason, not the card.
What is unproven until then
That the T-Head partition path produces any correct figure at all. This is the least-verified code in
#92 and the PR says so. The shared half of the path is not in question — partitionTargetsOf,
withPartitions, Figures and Resolve are manufacturer-agnostic and were exercised on an H100 — but
every HGML call in the new file is unexercised, including whether MigDevice.GetMemoryInfoV() answers
on a PPU partition handle at all.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read pkg/devicemanager/detector/thead/mig_process.go and run the verification on a T-Head PPU Kubernetes node with MIG enabled. Restart the device-manager DaemonSet, inspect the partition subresource, exporter, and /monitor/snapshot, and run cases/case-39.sh for surrounding claims. Done means the partition reports its own UUID, capacity, usage, and unsupported cores reason without inheriting parent-card figures; also check the no-identifier path if possible.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- infrastructure, observability, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100