NVIDIA / NVIDIA/NemoClaw

Project scenario-selected services into experimental NemoCUA sandboxes

Open
#10,289 0 comments 0 reactions 0 assignees View on GitHub
area: policy area: sandbox CUA experimental v0.0.127
Dominant language
TypeScript
Stars
22.5k
Forks
3.1k
Avg merge
1d 1h
Merged PRs (30d)
715

Description

## Problem statement

The experimental NemoCUA terminal-agent path accepts a caller-prepared image, but its baseline policy exposes only `inference.local`. A prepared image cannot run one browser task when its scenario-owned browser, computer, terminal, and synthetic fixture services run across the sandbox boundary.

The prepared image also requires `/app/config/config.toml`. NemoClaw does not project host-selected service endpoints into that file. Embedding host authority in the portable scenario would couple the image to one worker and bypass NemoClaw's network lifecycle ownership.

## Desired behavior

With `NEMOCLAW_CUA_ENABLED=1` and an explicit local prepared image, `nemoclaw onboard --agent nemocua` must:

- accept host-adapter-selected endpoints for exactly the browser, computer, terminal, and synthetic fixture services;
- project sandbox-facing endpoint values into `/app/config/config.toml` without putting private host coordinates in the image;
- grant the prepared image network access only to those four endpoints and `inference.local`;
- preserve the ordinary terminal-agent `agent start`, sandbox recovery, and destroy paths;
- reject missing, malformed, duplicate, or unsupported endpoint input before sandbox creation.

The consumer validates one bounded browser task and exact synthetic fixture state on the approved worker.

## Scope and exclusions

NemoClaw owns the public manifest, policy and config projection, and sandbox network lifecycle.

This issue does not add artifact transfer, image composition, fixture behavior, provider provisioning, first-class CUA APIs, nested NemoCUA sandboxes, target lifecycle state, or task lifecycle state.

The private scenario remains responsible for preparing the exact image, selecting and running the four services, choosing the bounded task, and verifying fixture state independently.

## Decision

- **Decision:** Accept
- **Reason and placement:** Core NemoClaw, inside the existing experimental, default-off NemoCUA terminal-agent path established by #9649 and merge `f47f8d552038ef7af8f5378ca57794af8535ca62`. The change fills the missing NemoClaw-owned sandbox boundary without creating a first-class CUA product API.
- **Accountable maintainer:** @jyaunches
- **Validation plan:** Focused source tests for endpoint input validation, exact policy materialization, config projection, disabled-gate behavior, and cleanup. Run CLI type-check and the routine PR validation gate. The private scenario owns one approved-worker consumer run and exact fixture-state verification.

## Compatibility requirements

The initial consumer tuple is:

- NemoClaw CLI `0.0.20-dev-v3`
- scenario services `0.0.66-dev-v29`
- NVLumina `v0.0.5`
- manifest digest `sha256:c1a577fc8f69071642b97706130df26abd8a89b8bd429a9ef37abf0ccd634e0b`

This tuple is consumer validation evidence, not a NemoClaw-managed release or compatibility registry.

## Security and privacy impact

The change crosses the sandbox-to-host network boundary. NemoClaw must accept only bounded, non-secret endpoint metadata; map it to one sandbox-facing host authority; restrict policy by exact ports, protocols, methods, and paths where the service contract permits; and prevent endpoint values from reaching logs, committed files, or durable scenario-independent state. No credentials are part of this projection.

## Acceptance evidence

- Enabled NemoCUA with one explicit prepared image and four valid host-selected service endpoints onboards through the ordinary agent path.
- The sandbox config names only the sandbox-facing endpoints.
- The effective policy contains `inference.local` and exactly the four selected tool-service destinations.
- Missing, malformed, duplicate, extra, credential-bearing, or non-local endpoint input fails before sandbox creation.
- The ordinary headless command receives `--task-id --query ` without a CUA-specific command layer.
- The private consumer proves exact fixture state and no nested sandbox.
- `nemoclaw destroy` removes the sandbox and its projected endpoint state.

Contributor guide

Open the contributing guide

Research direction

Start in the experimental, default-off NemoCUA terminal-agent path established by #9649 and commit f47f8d552038ef7af8f5378ca57794af8535ca62; trace the existing onboard, agent start, recovery, and destroy flows. Add focused source tests for endpoint validation, policy materialization, config projection, disabled-gate behavior, and cleanup, then run the CLI type-check and routine PR validation. Done means the ordinary path accepts only the four bounded endpoints plus inference.local and rejects invalid input before sandbox creation.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
cli, networking, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.