CI gap: builders & mirror paths not validated against real apiserver schemas
- Dominant language
- Rust
- Stars
- 42
- Forks
- 10
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 19
Description
## Summary
Three launch-eve bugs (#226, #227, #229) all shipped past CI because no test runs CLI/controller-emitted Kubernetes objects through real apiserver validation.
## What we missed
| PR | Bug | Why CI missed it |
|----|-----|------------------|
| #226 | `ToolPolicy.spec.appliesTo.sandboxName` (field doesn't exist) | Tests asserted JS object shape against itself; no CRD-OpenAPI schema check |
| #227 | `InferencePolicy.spec.modelPreference.primary.endpoint` (field doesn't exist; strict decoder rejects) | Same as above; `buildInferencePolicy` not run through any apiserver |
| #229 | Mirror ConfigMap label *value* contained `/` (K8s rejects) | E2E covers ToolPolicy creation + ClawSandbox creation in isolation, but the e2e `ClawSandbox` has no `toolPolicyRef` so `mirror_configmap` is never invoked |
## Fixes
### 1. CRD-OpenAPI validator in `cli/test` (~50 LOC)
Load `deploy/helm/azureclaw/templates/crd-*.yaml`, extract OpenAPI schemas, run every `build*` function in `cli/src/refs.ts` and `cli/src/migrate/*` through ajv. Catches #226 and #227 at PR time without needing a cluster.
### 2. Wire `toolPolicyRef` into the e2e ClawSandbox
In `tests/e2e/run.sh:153` (`test_create_sandbox`):
- Apply a ToolPolicy alongside the InferencePolicy
- Add `toolPolicyRef: { name: e2e-test-toolpolicy }` to the ClawSandbox spec
- After namespace appears, wait for the mirrored ConfigMap `toolpolicy-e2e-test-toolpolicy-profile` in `azureclaw-e2e-test` ns
This exercises `mirror_configmap` and `mirror_secret` against a real apiserver — would have caught #229.
### 3. Fix the 3 deferred audit findings (separate PR)
From the earlier session audit:
- `from_kagent.ts:463` — `allowedEndpoints[*].port` missing (default to 443)
- `toolpolicy.ts:43-47` — CLI doesn't fail-fast on empty `sandboxMatchLabels` (kubectl rejects via CEL)
- `convert.ts:515-525` + `migrate.ts:116-120` — `spec.upstreamCompatibility` field doesn't exist in ClawSandbox CRD (overlay/translate/observe modes silently broken)
The validator from (1) catches all three automatically.
## Priority
Post-launch. None of these block today's release; they are CI hardening to prevent the same class of regression.
Contributor guide
Research direction
Start in cli/test, deploy/helm/azureclaw/templates/crd-*.yaml, cli/src/refs.ts, and cli/src/migrate/*. Run the existing CLI builder tests and inspect tests/e2e/run.sh at test_create_sandbox around line 153. Done means emitted objects are checked against CRD schemas and the e2e path validates ToolPolicy mirroring through a real apiserver.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- helm, kubernetes, typescript
- Domain
- ci-cd, cli, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 58/100