anthonyoteri / anthonyoteri/cargo-rigtest
Profiles: named target environments with per-test selection
- Ngôn ngữ chính
- Rust
- Star
- 0
- Fork
- 0
- Merge trung bình
- 3 ngày 18 giờ
- Pull request đã merge (30 ngày)
- 8
Mô tả
## Problem
Acceptance tests target deployed systems, and the same suite often needs to run against multiple deployments — dev, staging, prod, regional variants, customer-specific tenants. Today every user who hits this builds their own ad-hoc solution: hardcoded URLs, environment-variable juggling, branching test code, separate test binaries per environment. There's no shared vocabulary or mechanism for "this test runs against staging" or "these credentials apply to prod."
This is the largest gap between rigtest and the "batteries-included for acceptance testing" identity. It's also the load-bearing foundation for several other planned features (diff mode, probe mode, the config home for trace deeplinks, and the profile-gated execution policy for destructive tests).
## Proposed solution
Introduce **profiles** — named target environments defined in a `rigtest.toml` config file at the workspace root. Each profile carries the configuration needed to point the suite at one deployment: environment variables, secrets sourcing, destructive-write policy, and (eventually) observability deeplink templates.
Config sketch:
```toml
default = "staging"
profiles = ["staging", "prod"] # authoritative list of active profiles
[profile.defaults] # merged into every profile
env_from = [".env"]
[profile.staging]
description = "internal staging"
env_from = [".env.staging"]
[profile.staging.environment]
TARGET_URL = "https://staging.example.com"
[profile.prod]
description = "production"
destructive = false
[profile.prod.environment]
TARGET_URL = "https://api.example.com"
```
Tests can pin themselves to one or more profiles:
```rust
#[testcase(profiles = ["staging", "prod"])]
async fn homepage_returns_200(ctx: Arc) -> Result<(), BoxError> {
// runs once per profile; reported as
// homepage_returns_200[profile=staging]
// homepage_returns_200[profile=prod]
}
```
The CLI flag filters within the declared set:
```
cargo rigtest run # default profile, all tests valid for it
cargo rigtest run --profile prod # narrows to tests valid against prod
```
**Selection rule:** *effective profiles = (declared profiles OR \[default]) ∩ (--profile filter OR all profiles)*. `--profile` always narrows; it never expands beyond what a test declared itself valid against.
## Alternatives considered
Users can do all of this by hand with environment variables, `dotenv`, and CLI wrappers — and many do. The cost is that every project rebuilds the same machinery, no two implementations share a vocabulary, and features that *would* depend on a structured notion of "what environment is this run targeting" (diff mode, probe mode, profile-pinned tests, destructive gating) can't be built portably.
A second alternative is to keep rigtest agnostic and ship only the per-test attribute, leaving config to the user. This fails to deliver on the "batteries-included" identity — the http-client feature exists precisely because making users wire up basic things themselves is the friction rigtest is built to remove.
## Additional context
This issue tracks the high-level feature. A detailed PRD will follow covering the points already discussed and the open questions that remain.
**Decided in discussion:**
- Naming: profiles, not "environments" (avoids overloading with OS env vars; matches cargo's `[profile.*]` precedent)
- File: `rigtest.toml` at workspace root (visible, not dotfile); follow cargo's walk-up discovery
- `profiles = [...]` is authoritative for the active set; `[profile.X]` blocks without a list entry are defined-but-inactive (preserves one-line comment-out for disabling)
- `[profile.defaults]` merges into every profile; "later wins" layering throughout
- Environment table is a sub-table (`[profile.X.environment]`), not a list of `KEY=VAL` strings
- `env_from = [...]` for dotenv-style files (secrets out of source control); paths relative to `rigtest.toml`
- `RIGTEST_*` is the single reserved env-var prefix; everything else (PATH, HOME, etc.) is fair game
- Strict validation: names in `profiles` must have corresponding `[profile.X]` blocks; orphan blocks warn rather than error
- Selection rule (intersection) for the per-test `profiles` attribute interacting with `--profile`
- Destructive policy on a profile (see #46) is optional and defaults to `true` (writes allowed) for backwards-compatible introduction
**Open questions for the PRD:**
- Variable interpolation (`TARGET_URL = "https://${HOST}.example.com"`) — defer to v2, but ensure literal `${...}` doesn't silently pass through
- Inheritance beyond `[profile.defaults]` — is `extends = "other"` needed, or does defaults cover the common case?
- Local override file (`.rigtest.local.toml`) — convention or first-class mechanism?
- Empty-intersection behavior on `--profile foo` when no test matches: clean exit with `0 matched` summary, or hard error?
- Trace deeplink template field (#43) — placeholder syntax (`{trace_id}` vs `${trace_id}` vs other)
- Whether `rigtest.toml` is required or optional (suites that don't use profiles should continue to work with no config file)
**Related issues:** #40 (`#[non_exhaustive]` — affects how profile-related fields can be added to `TestCase` without breaking releases), #43 (trace deeplinks — config home is per-profile), #41 (tags — orthogonal but composes with profile-pinned tests for filtering), #46 (destructive markers — gated by per-profile policy)
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.