NVIDIA / NVIDIA/NemoClaw

[Epic] Support computer-use agents as a first-class agent type

Open
#6,015 3 comments 0 reactions 1 assignee Claimed by @jyaunches View on GitHub
area: sandbox area: skills CUA needs: design
Dominant language
TypeScript
Stars
22.5k
Forks
3.1k
Avg merge
1d 1h
Merged PRs (30d)
715

Description

## Decision

#9649 and merged PR #9657 replace the original CUA candidate and qualification design.

The current implementation is one experimental, default-off terminal agent. This epic tracks that foundation and any later first-class CUA capabilities that maintainers accept.

Candidate status, qualification gates, release manifests, readiness records, component receipts, and CUA-specific target or task state are not current requirements.

## Current foundation

- `NEMOCLAW_CUA_ENABLED=1` is the only value that exposes `nemocua`.
- `NEMOCLAW_CUA_SANDBOX_IMAGE_REF` selects the caller-prepared sandbox image.
- NemoClaw validates the image-reference syntax and uses the image as the sandbox base image.
- The `agents/nemocua` agent manifest defines terminal commands, configuration paths, smoke checks, and baseline policy additions.
- `nemoclaw launch ` uses the existing interactive terminal path.
- `nemoclaw agent` runs the NemoCUA harness through the existing agent path.
- Existing onboarding, status, doctor, managed inference, policy, recovery, rebuild, and destroy behavior remains authoritative.
- NemoClaw does not select a NemoCUA release, resolve private artifacts, or attest the supplied image contents.
- This path is experimental. It does not establish product support or a compatibility promise.

## Ownership

NemoClaw owns:

- the exact default-off feature gate;
- agent discovery and selection;
- image-reference validation and sandbox handoff;
- one OpenShell-managed sandbox;
- the baseline filesystem, process, and network policy;
- managed inference;
- the existing sandbox lifecycle.

The active scenario or caller owns:

- image preparation, distribution, contents, and version selection;
- compatibility between the image, harness, and desktop target;
- desktop-target provisioning, transport, reset, and removal;
- target credentials, task input, task orchestration, and private artifacts;
- independent result verification and scenario-resource cleanup.

NemoClaw core remains environment-agnostic. It creates no nested NemoCUA sandbox and stores no CUA-specific target, security, task, or evidence state.

## Current demo path

The private browser demo can proceed on the merged foundation:

1. The scenario prepares the NemoCUA sandbox image.
2. The caller sets the feature gate and image reference.
3. The caller onboards `nemocua` through the standard agent path.
4. The scenario invokes the harness through `nemoclaw agent`.
5. The scenario owns the desktop target, task flow, independent verification, and cleanup.

The scenario repository owns its Brev run and browser-task acceptance. The demo does not depend on a first-class NemoClaw target or task API.

## Deferred first-class work

Do not implement the deferred issues from their historical bodies. Re-refine each issue against the standard terminal-agent foundation before implementation.

1. #8450: First determine whether the standard agent and policy commands can connect one caller-managed disposable target. Add no CUA target subsystem unless a current requirement proves that these commands are insufficient.
2. A future first-class browser-task boundary requires a new accepted issue after #8450 establishes the required target boundary.
3. #7751: Define terminal-task recovery only after an accepted shared task boundary exists.
4. #7752: Define the computer and integrated journeys only after the earlier lifecycle work exists.
5. #7753: Re-scope only if maintainers resume a product release-qualification decision. It is not a feature slice.

Related records:

- #7750 preserves the environment-agnostic architecture constraint. Its larger design is historical context.
- #7754 is historical security input. Map only applicable safeguards into each accepted capability.
- #7755 is superseded and is not a dependency for current or future work.

## Refinement requirements

Before implementation starts on a deferred capability, its issue must:

- identify one current consumer and one independently useful outcome;
- establish product ownership, lifecycle, compatibility, security, and validation;
- reuse the standard agent, policy, and sandbox paths unless a current requirement prevents it;
- define credential custody, failure, interruption, recovery, and cleanup behavior;
- select deterministic tests and the minimum required live boundary;
- remove #7755 and all candidate, qualification, readiness, digest, and receipt assumptions.

Do not add a second CUA feature gate, a parallel agent lifecycle, duplicated sandbox state, or a general target or task framework.

## Completion criteria

- [x] #9649 defines the experimental terminal-agent foundation.
- [x] #9657 implements that foundation on `main`.
- [x] The private demo has no dependency on the deferred first-class issues.
- [ ] After the private demo, maintainers decide whether NemoClaw stops at the experimental foundation or resumes first-class product work.
- [ ] If product work resumes, maintainers accept a refined #8450 before implementation.
- [ ] Maintainers record any final enablement and product-support decision.
- [ ] If product work does not resume, close the deferred issues without describing the experimental foundation as supported.

## Non-goals

- General remote-desktop or target-fleet management.
- Brev control-plane behavior in NemoClaw core.
- Employee desktops, production accounts, or shared mutable desktop targets.
- A universal CUA abstraction.
- Cross-agent delegation, messaging, or dashboard work.
- Public storage of task content, screenshots, documents, browser state, credentials, or transport data.
- New CUA-specific state or APIs without an accepted capability that consumes them.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.