alibaba / alibaba/ROCK

Support first-class GPU allocation for ROCK sandboxes

Open
#657 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
485
Forks
81
Avg merge
16h 12m
Merged PRs (30d)
8

Description

## Title

Support first-class GPU allocation for ROCK sandboxes

## Problem

ROCK can start sandbox containers, but GPU access is not currently a first-class concept across the full stack. In practice, users who need GPU-bound execution inside sandboxes have to patch the server-side Docker launch path manually.

This is limiting for:

- agentic evaluation where tools inside the sandbox need CUDA
- GPU-accelerated code execution or tests inside sandboxed repos
- mixed CPU/GPU sandbox fleets
- deterministic per-sandbox GPU assignment in multi-sandbox runs

## Requested Capability

Add end-to-end GPU support for sandboxes across:

- SDK request model
- admin API
- runtime deployment layer
- scheduler / placement layer
- operator-specific backends

## Proposed API Shape

Examples of the sort of fields that would be useful:

- `enable_gpu_passthrough: bool`
- `gpu_count: int | None`
- `gpu_device_request: str | None`
- `gpu_allocation_mode: Literal["fixed", "round_robin"]`

These should ideally be available:

- in SDK `SandboxConfig`
- in admin `SandboxStartRequest`
- in deployment config objects

## Expected Behavior

- request `all` GPUs or a specific count
- optionally request explicit device ids
- respect pre-existing `docker_args` / operator overrides
- support deterministic multi-sandbox allocation
- fail clearly when host GPU runtime is unavailable
- expose the effective GPU assignment in sandbox status / logs

## Backend Considerations

### Docker

- map request to `docker run --gpus ...`
- set appropriate visibility env vars when assignment is specific

### Ray

- reserve GPU-capable placement resources, not just CPU/memory

### Kubernetes

- map requests into pod resource requests/limits or template selection

## Why This Matters

Without first-class support, local patches can make sandbox GPU behavior work in one deployment but not in a portable or upstreamable way. A supported API would let ROLL and other ROCK users request GPU-capable sandboxes predictably and safely.

## Current Workaround

A server-side workaround can be implemented by extending ROCK runtime config and Docker launch logic, but that still leaves the SDK/API and scheduler layers unaware of GPU requirements. That is useful as an interim step, but not a complete solution.

Contributor guide

No contributing guide indexed for this repository

Research direction

Trace GPU requirements from SDK SandboxConfig and admin SandboxStartRequest through deployment config, the Docker launch path, scheduler or placement logic, and operator-specific backends. Compare the Docker, Ray, and Kubernetes considerations in the issue; done means requests are propagated end to end, allocation is deterministic, unavailable runtimes fail clearly, and effective assignments appear in status or logs.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, kubernetes, python
Domain
backend, cloud, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.