NVIDIA / NVIDIA/OpenShell

test(e2e): separate VM gateway provisioning from test execution

Open
#2,314 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

state:stale
Dominant language
Rust
Stars
8.7k
Forks
1.3k
Avg merge
2d 11h
Merged PRs (30d)
253

Description

Description

Introduce e2e/with-vm-gateway.sh as the VM equivalent of the existing Docker, Podman, and Kubernetes gateway provisioners. The script should own VM gateway setup and teardown while accepting an arbitrary command to run against the provisioned gateway.

Refactor e2e/rust/e2e-vm.sh into a thin test-selection wrapper, or replace it with task-level commands, so provisioning no longer hard-codes smoke and API conformance test execution.

Context

The current e2e/rust/e2e-vm.sh combines several responsibilities:

  • preparing the embedded VM runtime and supervisor
  • building and codesigning VM artifacts
  • generating PKI and gateway configuration
  • starting and waiting for the gateway
  • selecting and running Rust test targets
  • collecting logs and tearing down the gateway

This differs from e2e/with-docker-gateway.sh, e2e/with-podman-gateway.sh, and e2e/with-kube-gateway.sh, which provision a gateway and then execute a caller-supplied command. The coupling currently requires VM smoke and API conformance to be run through separate hard-coded Cargo invocations.

PR #2305 adds VM smoke, host-gateway, and gateway-resume coverage to CI and should be considered when preserving the supported VM test matrix. The API/CLI conformance work associated with #2182 also benefits from a uniform provisioner contract.

Proposed Design

  • Add e2e/with-vm-gateway.sh <command> [args...].
  • Move VM runtime preparation, artifact builds, macOS codesigning, ephemeral state, PKI generation, gateway startup/readiness, diagnostic log collection, and teardown into the provisioner.
  • Export the same connection contract consumed by portable test suites, including OPENSHELL_GATEWAY, OPENSHELL_GATEWAY_ENDPOINT, and the OPENSHELL_CONFORMANCE_TLS_* paths.
  • Preserve VM-specific context such as OPENSHELL_E2E_DRIVER, OPENSHELL_E2E_VM_STATE_DIR, restart metadata, provisioning timeouts, and overlay expectations where required.
  • Propagate the child command's exit status while always performing cleanup and failure diagnostics.
  • Make VM test selection explicit in tasks and CI, rather than inside the provisioner.
  • Update PR #2305's workflow paths or their successors to use the new wrapper for smoke, host-gateway, resume, and conformance coverage.

Definition of Done

  • e2e/with-vm-gateway.sh provisions one VM-backed gateway and runs an arbitrary caller-supplied command.
  • Gateway teardown and VM/gateway diagnostic collection occur on success, failure, and interruption.
  • Existing local VM smoke and gateway-resume tasks continue to pass.
  • The VM CI coverage introduced by PR #2305 continues to run without losing scenarios.
  • API conformance can run against the VM gateway without a VM-specific hard-coded Cargo invocation.
  • VM test selection is owned by tasks/workflows or thin test wrappers, separately from provisioning.
  • TESTING.md, CI.md, and relevant architecture documentation describe the resulting workflow.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Compare e2e/rust/e2e-vm.sh with e2e/with-docker-gateway.sh, e2e/with-podman-gateway.sh, and e2e/with-kube-gateway.sh to map the provisioning contract. Review PR #2305's workflow paths and the VM smoke, resume, and conformance tasks, then update TESTING.md, CI.md, and relevant architecture documentation. Done means VM provisioning is reusable, cleanup and diagnostics run on every outcome, and all listed VM coverage still runs.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, shell
Domain
ci-cd, documentation, infrastructure, testing
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.