feat: E2E and release testing improvements
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 8.7k
- Forks
- 1.3k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 253
Description
Problem Statement
Today it is difficult to assemble reproducible E2E environments with the correct operating-system dependencies and configuration, such as Fedora with Podman, Rocky Linux with Docker, or an SELinux-enabled host. We want to make it easy to provision these environments, deploy OpenShell into them, and run E2E and release tests against the resulting deployment across documented beta platforms.
Proposed Design
-
1. Common test guest foundation (#2471). Land reusable Nix/QEMU guests with an uncached correctness path and optional OCI-backed QCOW2 caching. Restrict cache publication to trusted workflows and fall back safely when caching fails.
-
2. VM-backed E2E runner (#2473). Build OpenShell from the checkout, run it inside a test guest, execute host-side E2E scenarios, and retain useful diagnostics. Clean up reliably after successful, failed, and interrupted runs.
-
3. Release canaries on test guests (#2472). Run release canaries inside representative Ubuntu and Fedora-family guests using the same artifacts and installation paths users consume. Exercise cache hits and forced misses while reporting boot, installation, and test timing separately.
-
4. Release upgrade coverage. Test clean installation and previous-release-to-candidate upgrades for Debian and RPM packages, verifying versions, gateway health, persisted state, and sandbox operation. Cover Debian/Ubuntu amd64 and arm64, and macOS Apple Silicon.
-
5. Test guests as development sandboxes. Make prepared test-guest VM images consumable through the OpenShell sandbox lifecycle so developers can provision a selected distro, runtime, and security configuration and build or test OpenShell inside it. Define image provenance, writable workspace, persistence, resource, lifecycle, and cache invalidation boundaries.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the completed common guest foundation (#2471) and VM-backed E2E runner (#2473), then review the unchecked release canary, upgrade coverage, and development-sandbox goals. Define a focused scope around one of those areas; done means the relevant guest workflows run against the documented platforms, report useful results, and clean up reliably.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, rust
- Domain
- ci-cd, infrastructure, release, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100