test(e2e): separate VM gateway provisioning from test execution
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 8.7k
- Forks
- 1.3k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 253
Description
Description
Introduce e2e/with-vm-gateway.sh as the VM equivalent of the existing Docker, Podman, and Kubernetes gateway provisioners. The script should own VM gateway setup and teardown while accepting an arbitrary command to run against the provisioned gateway.
Refactor e2e/rust/e2e-vm.sh into a thin test-selection wrapper, or replace it with task-level commands, so provisioning no longer hard-codes smoke and API conformance test execution.
Context
The current e2e/rust/e2e-vm.sh combines several responsibilities:
- preparing the embedded VM runtime and supervisor
- building and codesigning VM artifacts
- generating PKI and gateway configuration
- starting and waiting for the gateway
- selecting and running Rust test targets
- collecting logs and tearing down the gateway
This differs from e2e/with-docker-gateway.sh, e2e/with-podman-gateway.sh, and e2e/with-kube-gateway.sh, which provision a gateway and then execute a caller-supplied command. The coupling currently requires VM smoke and API conformance to be run through separate hard-coded Cargo invocations.
PR #2305 adds VM smoke, host-gateway, and gateway-resume coverage to CI and should be considered when preserving the supported VM test matrix. The API/CLI conformance work associated with #2182 also benefits from a uniform provisioner contract.
Proposed Design
- Add
e2e/with-vm-gateway.sh <command> [args...]. - Move VM runtime preparation, artifact builds, macOS codesigning, ephemeral state, PKI generation, gateway startup/readiness, diagnostic log collection, and teardown into the provisioner.
- Export the same connection contract consumed by portable test suites, including
OPENSHELL_GATEWAY,OPENSHELL_GATEWAY_ENDPOINT, and theOPENSHELL_CONFORMANCE_TLS_*paths. - Preserve VM-specific context such as
OPENSHELL_E2E_DRIVER,OPENSHELL_E2E_VM_STATE_DIR, restart metadata, provisioning timeouts, and overlay expectations where required. - Propagate the child command's exit status while always performing cleanup and failure diagnostics.
- Make VM test selection explicit in tasks and CI, rather than inside the provisioner.
- Update PR #2305's workflow paths or their successors to use the new wrapper for smoke, host-gateway, resume, and conformance coverage.
Definition of Done
-
e2e/with-vm-gateway.shprovisions one VM-backed gateway and runs an arbitrary caller-supplied command. - Gateway teardown and VM/gateway diagnostic collection occur on success, failure, and interruption.
- Existing local VM smoke and gateway-resume tasks continue to pass.
- The VM CI coverage introduced by PR #2305 continues to run without losing scenarios.
- API conformance can run against the VM gateway without a VM-specific hard-coded Cargo invocation.
- VM test selection is owned by tasks/workflows or thin test wrappers, separately from provisioning.
-
TESTING.md,CI.md, and relevant architecture documentation describe the resulting workflow.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Compare e2e/rust/e2e-vm.sh with e2e/with-docker-gateway.sh, e2e/with-podman-gateway.sh, and e2e/with-kube-gateway.sh to map the provisioning contract. Review PR #2305's workflow paths and the VM smoke, resume, and conformance tasks, then update TESTING.md, CI.md, and relevant architecture documentation. Done means VM provisioning is reusable, cleanup and diagnostics run on every outcome, and all listed VM coverage still runs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, shell
- Domain
- ci-cd, documentation, infrastructure, testing
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100