[Epic] Validate v1 against representative v0 deployments
- Dominant language
- TypeScript
- Stars
- 22.5k
- Forks
- 3.1k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 715
Description
## Problem Statement
NemoClaw v1 needs deployment-level evidence that representative deployments available in v0 can be reproduced through the v1 desired-state SDK, CLI, and OpenTofu provider.
Existing v1 tests qualify individual contracts and several complete lifecycles, but they do not provide a tracked comparison of v0 deployments against the closest v1 desired states derived from v0 exports.
## Desired Behavior
For each selected v0 deployment:
1. Identify the exact v0 CLI or release artifact used, including its version, revision when available, and executable or artifact hash.
2. Record redacted v0 inputs, platform, runtime, agent, inference provider, model, policy, integrations, and external dependencies.
3. Use the public v0 CLI to create and verify an owned deployment.
4. Export its secret-free configuration with the public v0 `config export` command before destroying it.
5. Translate that export into the closest supported v1 desired state, adding only explicit v1-only runtime bindings.
6. Run v1 plan, apply, real-agent verification, unchanged apply, export/reapply, and destroy.
7. Retain redacted evidence and classify the result.
The harness should be owned by v1 and treat v0 as a black-box CLI. It should not patch or import v0 test internals. A signed v0 release tag or installed release that supports `config export` is sufficient when its exact identity is recorded; a particular source commit is not a permanent scenario requirement.
A scenario result is one of:
- Equivalent.
- Equivalent with an intentional architectural difference.
- Missing v1 product surface.
- v1 behavioral defect.
- Not qualified because of an environment or external dependency.
## Scenario Checklist
- [ ] OpenClaw with an NVIDIA hosted inference endpoint on Linux/Docker
- [ ] Hermes with an NVIDIA hosted inference endpoint
- [ ] LangChain Deep Agents Code with an NVIDIA hosted inference endpoint
- [ ] OpenClaw with local Ollama
- [ ] OpenClaw with managed vLLM on DGX Spark
- [ ] OpenClaw with an explicit or custom network policy
- [ ] OpenClaw with a remote vLLM service over SSH
- [ ] Messaging integration deployment
- [ ] Managed MCP server deployment
- [ ] Snapshot, restore, or rebuild lifecycle
Create focused child issues when a scenario requires implementation, dedicated hardware, external credentials, or substantial investigation.
## Evidence Required Per Scenario
- Exact v0 CLI/release identity and v1 revision.
- Redacted v0 inputs, exact v0 export bytes and hash, translated v1 input, and v1 export.
- Explicit v1-only bindings and a record of fields changed or not representable during translation.
- Host platform, architecture, runtime, and relevant hardware.
- Immutable image, model, bundle, and dependency identities where applicable.
- Commands and observable results.
- Successful real agent request when the scenario includes inference.
- Resource identities across unchanged apply and export/reapply.
- v0 and v1 destroy results and retained-resource behavior.
- Differences, limitations, failures, and final classification.
Passing configuration parsing or compilation alone does not establish deployment parity.
## Scope and Exclusions
Included:
- Representative deployments supported or realistically possible in v0.
- The closest supported v1 equivalent derived from a real v0 export.
- Functional, lifecycle, ownership, recovery, and credential-boundary comparisons.
- Evidence that proves either parity or a concrete gap.
- A reusable, data-driven v1 harness that can exercise multiple v0 configurations through public CLI behavior.
Excluded:
- Importing v0 runtime state directly into v1.
- Patching v0 source or depending on a particular v0 E2E fixture as the migration interface.
- Requiring identical configuration syntax or internal resource graphs.
- Claiming exhaustive compatibility from a representative scenario.
- Treating an unsupported v1 feature as a defect without a scope decision.
- Using unowned live resources or exposing credentials in evidence.
## Ongoing Ownership
@cv owns the v1 scope and acceptance boundary.
Each scenario contributor owns its test resources, cleanup, redaction, and evidence until the scenario is accepted or handed off.
## Placement and Support Expectations
Core NemoClaw v1 validation work.
Results should be retained with the existing v1 validation evidence and linked from this issue. Confirmed implementation gaps should receive focused issues rather than expanding a validation scenario indefinitely.
The baseline is native Linux with a Docker daemon. Architecture is an observed environment property, not a required compatibility matrix unless a scenario specifically needs one. Docker Desktop findings may inform a separate support decision but do not replace native-Linux qualification.
## Validation Plan
Each completed scenario must exercise the real deployment boundary appropriate to that case.
The v0 lifecycle should include:
- Preflight proving the scenario-owned identity is absent.
- Normal CLI onboarding or creation.
- Readiness and a real agent request where applicable.
- Normal CLI `config export` while the deployment remains registered.
- Normal CLI destroy with owned-resource verification.
The v1 lifecycle should include:
- Translation from the exact v0 export with explicit v1-only bindings.
- Read-only plan.
- Apply and readiness.
- A real agent request where applicable.
- Unchanged apply.
- Export and reapply.
- Stable resource-identity comparison.
- Safe failure behavior for important negative cases.
- Destroy with owned-resource and retained-storage verification.
Deterministic fake-CLI fixtures should cover orchestration failures, cleanup, redaction, and proof publication, but they do not replace required live evidence.
## Compatibility Requirements
Record compatibility per scenario rather than making a branch-wide claim.
Start with Linux/Docker and hosted inference. Add platform, runtime, provider, and hardware combinations only when an owned environment is available.
## Security or Privacy Impact
Live scenarios may use external credentials, model endpoints, Docker or Podman daemons, OpenShell gateways, and GPU resources.
Use dedicated credentials and owned deployment identities. Do not place secret values in YAML, commands, logs, issues, or retained evidence. Prefer hidden interactive entry or an approved runtime secret mechanism. Revoke temporary credentials through their issuer after use.
The harness must register cleanup ownership before creation, fail closed rather than adopt an existing resource, and withhold a passing proof until both product lifecycles have destroyed their owned resources successfully.
## Related Work
- #10904 — declarative configuration epic
- #11813 — draft implementation of the first scenario and reusable translation/lifecycle contracts
- v1 accepted scope: `docs/design/scope.md`
- v1 validation matrix: `docs/validation/README.md`
- v0 E2E manifests and fixtures: `test/e2e/`
## Category
Testing
## Checklist
- [x] I searched existing issues and this is not a duplicate.
- [x] I described the problem and desired behavior.
Contributor guide
Assessment
This issue has not been assessed yet.