task: assess Kubernetes GPU support preflight validation
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 8.7k
- Forks
- 1.3k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 253
Description
Description
Assess how the Kubernetes compute driver should determine whether a cluster exposes GPU resources before accepting GPU sandbox requests.
This follows from the --gpu-count work and should not change behavior in the current small CLI PR. The goal is to decide what signal the driver should use for fail-fast feedback when a cluster has no GPU support, without incorrectly rejecting requests that Kubernetes could schedule later.
Context
Related roadmap: #1444
The current Kubernetes driver preflight checks node status.allocatable["nvidia.com/gpu"] and treats any non-zero value as GPU support. During review of the GPU count CLI work, we identified that this check is really about whether the cluster exposes GPU resources, not whether a particular sandbox can schedule immediately.
Important distinctions:
capacityis a more stable signal that a node advertises the GPU extended resource.allocatableis closer to whether pods can request that resource.- Neither field alone answers whether a specific pod can schedule right now.
- Current resource pressure should generally be left to the Kubernetes scheduler, because a sandbox may become schedulable after other pods release GPUs.
- Autoscaled GPU node pools may complicate any check based only on currently present nodes.
Definition of Done
- Decide whether the Kubernetes preflight should use node
capacity, nodeallocatable, both, or another signal. - Decide whether GPU support detection should remain boolean or account for requested
gpu_count. - Decide how zero-scaled or autoscaled GPU node pools should be handled.
- Document the intended semantics in the Kubernetes driver README or architecture docs.
- Create implementation follow-up issues if behavior should change.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the current Kubernetes driver preflight check that inspects node status.allocatable and the related --gpu-count work. Compare capacity, allocatable, requested gpu_count, and autoscaled or zero-scaled pools against the stated semantics. Document the decision in the Kubernetes driver README or architecture docs, and record implementation follow-up issues if needed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes
- Domain
- infrastructure
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100