Azure / Azure/kars

CI gap: builders & mirror paths not validated against real apiserver schemas

Open
#230 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
42
Forks
10
Avg merge
1d 11h
Merged PRs (30d)
19

Description

## Summary

Three launch-eve bugs (#226, #227, #229) all shipped past CI because no test runs CLI/controller-emitted Kubernetes objects through real apiserver validation.

## What we missed

| PR | Bug | Why CI missed it |
|----|-----|------------------|
| #226 | `ToolPolicy.spec.appliesTo.sandboxName` (field doesn't exist) | Tests asserted JS object shape against itself; no CRD-OpenAPI schema check |
| #227 | `InferencePolicy.spec.modelPreference.primary.endpoint` (field doesn't exist; strict decoder rejects) | Same as above; `buildInferencePolicy` not run through any apiserver |
| #229 | Mirror ConfigMap label *value* contained `/` (K8s rejects) | E2E covers ToolPolicy creation + ClawSandbox creation in isolation, but the e2e `ClawSandbox` has no `toolPolicyRef` so `mirror_configmap` is never invoked |

## Fixes

### 1. CRD-OpenAPI validator in `cli/test` (~50 LOC)

Load `deploy/helm/azureclaw/templates/crd-*.yaml`, extract OpenAPI schemas, run every `build*` function in `cli/src/refs.ts` and `cli/src/migrate/*` through ajv. Catches #226 and #227 at PR time without needing a cluster.

### 2. Wire `toolPolicyRef` into the e2e ClawSandbox

In `tests/e2e/run.sh:153` (`test_create_sandbox`):
- Apply a ToolPolicy alongside the InferencePolicy
- Add `toolPolicyRef: { name: e2e-test-toolpolicy }` to the ClawSandbox spec
- After namespace appears, wait for the mirrored ConfigMap `toolpolicy-e2e-test-toolpolicy-profile` in `azureclaw-e2e-test` ns

This exercises `mirror_configmap` and `mirror_secret` against a real apiserver — would have caught #229.

### 3. Fix the 3 deferred audit findings (separate PR)

From the earlier session audit:
- `from_kagent.ts:463` — `allowedEndpoints[*].port` missing (default to 443)
- `toolpolicy.ts:43-47` — CLI doesn't fail-fast on empty `sandboxMatchLabels` (kubectl rejects via CEL)
- `convert.ts:515-525` + `migrate.ts:116-120` — `spec.upstreamCompatibility` field doesn't exist in ClawSandbox CRD (overlay/translate/observe modes silently broken)

The validator from (1) catches all three automatically.

## Priority

Post-launch. None of these block today's release; they are CI hardening to prevent the same class of regression.

Contributor guide

Open the contributing guide

Research direction

Start in cli/test, deploy/helm/azureclaw/templates/crd-*.yaml, cli/src/refs.ts, and cli/src/migrate/*. Run the existing CLI builder tests and inspect tests/e2e/run.sh at test_create_sandbox around line 153. Done means emitted objects are checked against CRD schemas and the e2e path validates ToolPolicy mirroring through a real apiserver.

Written by the indexing model from the issue text.

Assessment

Tech stack
helm, kubernetes, typescript
Domain
ci-cd, cli, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.