anthonyoteri / anthonyoteri/cargo-rigtest

Profiles: named target environments with per-test selection

Đang mở
#47 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
enhancement
Ngôn ngữ chính
Rust
Star
0
Fork
0
Merge trung bình
3 ngày 18 giờ
Pull request đã merge (30 ngày)
8

Mô tả

## Problem

Acceptance tests target deployed systems, and the same suite often needs to run against multiple deployments — dev, staging, prod, regional variants, customer-specific tenants. Today every user who hits this builds their own ad-hoc solution: hardcoded URLs, environment-variable juggling, branching test code, separate test binaries per environment. There's no shared vocabulary or mechanism for "this test runs against staging" or "these credentials apply to prod."

This is the largest gap between rigtest and the "batteries-included for acceptance testing" identity. It's also the load-bearing foundation for several other planned features (diff mode, probe mode, the config home for trace deeplinks, and the profile-gated execution policy for destructive tests).

## Proposed solution

Introduce **profiles** — named target environments defined in a `rigtest.toml` config file at the workspace root. Each profile carries the configuration needed to point the suite at one deployment: environment variables, secrets sourcing, destructive-write policy, and (eventually) observability deeplink templates.

Config sketch:

```toml
default = "staging"
profiles = ["staging", "prod"] # authoritative list of active profiles

[profile.defaults] # merged into every profile
env_from = [".env"]

[profile.staging]
description = "internal staging"
env_from = [".env.staging"]
[profile.staging.environment]
TARGET_URL = "https://staging.example.com"

[profile.prod]
description = "production"
destructive = false
[profile.prod.environment]
TARGET_URL = "https://api.example.com"
```

Tests can pin themselves to one or more profiles:

```rust
#[testcase(profiles = ["staging", "prod"])]
async fn homepage_returns_200(ctx: Arc) -> Result<(), BoxError> {
// runs once per profile; reported as
// homepage_returns_200[profile=staging]
// homepage_returns_200[profile=prod]
}
```

The CLI flag filters within the declared set:

```
cargo rigtest run # default profile, all tests valid for it
cargo rigtest run --profile prod # narrows to tests valid against prod
```

**Selection rule:** *effective profiles = (declared profiles OR \[default]) ∩ (--profile filter OR all profiles)*. `--profile` always narrows; it never expands beyond what a test declared itself valid against.

## Alternatives considered

Users can do all of this by hand with environment variables, `dotenv`, and CLI wrappers — and many do. The cost is that every project rebuilds the same machinery, no two implementations share a vocabulary, and features that *would* depend on a structured notion of "what environment is this run targeting" (diff mode, probe mode, profile-pinned tests, destructive gating) can't be built portably.

A second alternative is to keep rigtest agnostic and ship only the per-test attribute, leaving config to the user. This fails to deliver on the "batteries-included" identity — the http-client feature exists precisely because making users wire up basic things themselves is the friction rigtest is built to remove.

## Additional context

This issue tracks the high-level feature. A detailed PRD will follow covering the points already discussed and the open questions that remain.

**Decided in discussion:**

- Naming: profiles, not "environments" (avoids overloading with OS env vars; matches cargo's `[profile.*]` precedent)
- File: `rigtest.toml` at workspace root (visible, not dotfile); follow cargo's walk-up discovery
- `profiles = [...]` is authoritative for the active set; `[profile.X]` blocks without a list entry are defined-but-inactive (preserves one-line comment-out for disabling)
- `[profile.defaults]` merges into every profile; "later wins" layering throughout
- Environment table is a sub-table (`[profile.X.environment]`), not a list of `KEY=VAL` strings
- `env_from = [...]` for dotenv-style files (secrets out of source control); paths relative to `rigtest.toml`
- `RIGTEST_*` is the single reserved env-var prefix; everything else (PATH, HOME, etc.) is fair game
- Strict validation: names in `profiles` must have corresponding `[profile.X]` blocks; orphan blocks warn rather than error
- Selection rule (intersection) for the per-test `profiles` attribute interacting with `--profile`
- Destructive policy on a profile (see #46) is optional and defaults to `true` (writes allowed) for backwards-compatible introduction

**Open questions for the PRD:**

- Variable interpolation (`TARGET_URL = "https://${HOST}.example.com"`) — defer to v2, but ensure literal `${...}` doesn't silently pass through
- Inheritance beyond `[profile.defaults]` — is `extends = "other"` needed, or does defaults cover the common case?
- Local override file (`.rigtest.local.toml`) — convention or first-class mechanism?
- Empty-intersection behavior on `--profile foo` when no test matches: clean exit with `0 matched` summary, or hard error?
- Trace deeplink template field (#43) — placeholder syntax (`{trace_id}` vs `${trace_id}` vs other)
- Whether `rigtest.toml` is required or optional (suites that don't use profiles should continue to work with no config file)

**Related issues:** #40 (`#[non_exhaustive]` — affects how profile-related fields can be added to `TestCase` without breaking releases), #43 (trace deeplinks — config home is per-profile), #41 (tags — orthogonal but composes with profile-pinned tests for filtering), #46 (destructive markers — gated by per-profile policy)

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.