Support first-class GPU allocation for ROCK sandboxes
- Dominant language
- Python
- Stars
- 485
- Forks
- 81
- Avg merge
- 16h 12m
- Merged PRs (30d)
- 8
Description
## Title
Support first-class GPU allocation for ROCK sandboxes
## Problem
ROCK can start sandbox containers, but GPU access is not currently a first-class concept across the full stack. In practice, users who need GPU-bound execution inside sandboxes have to patch the server-side Docker launch path manually.
This is limiting for:
- agentic evaluation where tools inside the sandbox need CUDA
- GPU-accelerated code execution or tests inside sandboxed repos
- mixed CPU/GPU sandbox fleets
- deterministic per-sandbox GPU assignment in multi-sandbox runs
## Requested Capability
Add end-to-end GPU support for sandboxes across:
- SDK request model
- admin API
- runtime deployment layer
- scheduler / placement layer
- operator-specific backends
## Proposed API Shape
Examples of the sort of fields that would be useful:
- `enable_gpu_passthrough: bool`
- `gpu_count: int | None`
- `gpu_device_request: str | None`
- `gpu_allocation_mode: Literal["fixed", "round_robin"]`
These should ideally be available:
- in SDK `SandboxConfig`
- in admin `SandboxStartRequest`
- in deployment config objects
## Expected Behavior
- request `all` GPUs or a specific count
- optionally request explicit device ids
- respect pre-existing `docker_args` / operator overrides
- support deterministic multi-sandbox allocation
- fail clearly when host GPU runtime is unavailable
- expose the effective GPU assignment in sandbox status / logs
## Backend Considerations
### Docker
- map request to `docker run --gpus ...`
- set appropriate visibility env vars when assignment is specific
### Ray
- reserve GPU-capable placement resources, not just CPU/memory
### Kubernetes
- map requests into pod resource requests/limits or template selection
## Why This Matters
Without first-class support, local patches can make sandbox GPU behavior work in one deployment but not in a portable or upstreamable way. A supported API would let ROLL and other ROCK users request GPU-capable sandboxes predictably and safely.
## Current Workaround
A server-side workaround can be implemented by extending ROCK runtime config and Docker launch logic, but that still leaves the SDK/API and scheduler layers unaware of GPU requirements. That is useful as an interim step, but not a complete solution.
Contributor guide
No contributing guide indexed for this repository
Research direction
Trace GPU requirements from SDK SandboxConfig and admin SandboxStartRequest through deployment config, the Docker launch path, scheduler or placement logic, and operator-specific backends. Compare the Docker, Ray, and Kubernetes considerations in the issue; done means requests are propagated end to end, allocation is deterministic, unavailable runtimes fail clearly, and effective assignments appear in status or logs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, kubernetes, python
- Domain
- backend, cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100