PRD: PlatformSettings and PlatformTeam CRDs with Go apl-operator

## Problem Statement

APL platform and Team settings are currently stored as loosely typed values and reconciled through a TypeScript operator, Helmfile, Git polling, and broad apply operations. Platform administrators lack stable Kubernetes resources for desired settings, reconciliation is coarsely scoped, schema ownership is duplicated, and `_rawValues` bypass typed contracts.

APL needs typed, auditable, event-driven configuration resources and a Go operator with explicit ownership, dependency, lifecycle, migration, and status contracts. Git must remain the durable source of user intent, while Argo CD remains responsible for workload deployment and healing.

## Solution

Introduce cluster-scoped `PlatformSettings` and `PlatformTeam` resources in `akamai.io/v1alpha1`, reconciled by a new Go `apl-operator` maintained in a separate repository.

`PlatformSettings` represents the single platform installation. `PlatformTeam` represents settings for one Team and uses its resource name as the immutable Team identity. Both resources expose strongly typed specs generated from authoritative Go types, reject unknown fields, omit `_rawValues`, and reference Secrets without containing secret material.

The API and console persist CR manifests to the values repository. Argo CD applies them. The operator watches live resources, computes defaults and derived values, and reconciles inline Helm values into operator-owned Argo CD Applications. It does not write watched specs or generated values back to Git and has no runtime dependency on the TypeScript CLI or Helmfile.

## Acceptance Criteria

1. **GIVEN** the CRDs are installed, **WHEN** `PlatformSettings` named `platform` has a valid spec, **THEN** Kubernetes accepts it.
2. **GIVEN** the CRDs are installed, **WHEN** `PlatformSettings` has another name, **THEN** CRD validation rejects it without a webhook.
3. **GIVEN** a valid `PlatformTeam`, **WHEN** Kubernetes accepts it, **THEN** `metadata.name` is its canonical immutable Team identity and no duplicate Team ID exists in spec.
4. **GIVEN** a `PlatformTeam`, **WHEN** `spec.displayName` changes, **THEN** presentation changes without changing Team identity.
5. **GIVEN** `PlatformSettings` is absent, **WHEN** a `PlatformTeam` reconciles, **THEN** it causes no deployment side effects and reports `Ready=False` with reason `PlatformSettingsNotFound`.
6. **GIVEN** that pending Team, **WHEN** `PlatformSettings` appears, **THEN** the Team is automatically enqueued.
7. **GIVEN** either CR contains an unknown field, `_rawValues`, a generic patch, or an arbitrary merge, **WHEN** it is validated, **THEN** it is rejected.
8. **GIVEN** an inherited optional setting, **WHEN** it is omitted or explicitly set to false or zero, **THEN** the operator preserves those distinct meanings.
9. **GIVEN** effective Team settings, **WHEN** an allowed Team override exists, **THEN** precedence is Team override, platform setting, then operator default.
10. **GIVEN** a forbidden Team override, **WHEN** it is submitted, **THEN** it is rejected or reported without mutating generated Applications.
11. **GIVEN** supported legacy platform settings, **WHEN** migrated, **THEN** their semantics are preserved in typed `PlatformSettings.spec` fields.
12. **GIVEN** supported legacy Team settings and typed Team app settings, **WHEN** migrated, **THEN** each Team becomes one `PlatformTeam` without a redundant settings ID.
13. **GIVEN** Team services, workloads, builds, code repositories, network policies, users, or memberships, **WHEN** the new CRs are generated, **THEN** those independently owned entities are not embedded.
14. **GIVEN** a Team has an external identity-provider group, **WHEN** represented in `PlatformTeam`, **THEN** only a typed external group identifier is stored, not membership data.
15. **GIVEN** a setting needs a credential, **WHEN** authored in either CR, **THEN** it stores only a typed Secret name and key reference, never secret material.
16. **GIVEN** a platform Secret reference, **WHEN** resolved, **THEN** it is restricted to `apl-secrets`; a Team reference is restricted to the derived primary Team namespace.
17. **GIVEN** a referenced Secret or key is missing or invalid, **WHEN** reconciliation runs, **THEN** last-known-good Applications remain unchanged and status identifies the reference.
18. **GIVEN** the Secret becomes valid, **WHEN** its event is observed, **THEN** the owning CR automatically retries.
19. **GIVEN** user-authored desired state, **WHEN** persisted, **THEN** Git contains one global PlatformSettings manifest and one separate global manifest per PlatformTeam.
20. **GIVEN** defaults, derived values, or generated Helm values, **WHEN** computed, **THEN** they are not written to Git or CR specs.
21. **GIVEN** valid desired state, **WHEN** reconciled, **THEN** the operator creates or updates Argo CD Applications with typed inline Helm values and Argo CD deploys and heals workloads.
22. **GIVEN** an existing Application with approved previous-operator ownership, **WHEN** cutover occurs, **THEN** it may be adopted; an unapproved collision remains unchanged and reports `OwnershipConflict`.
23. **GIVEN** operator-owned Applications, **WHEN** applied, **THEN** Server-Side Apply uses distinct platform and Team field managers and does not force conflicts.
24. **GIVEN** a PlatformTeam change, **WHEN** processed, **THEN** only that Team is enqueued.
25. **GIVEN** a PlatformSettings change, **WHEN** dependencies are evaluated, **THEN** only Teams whose effective settings depend on changed fields are enqueued.
26. **GIVEN** an affected Team has an unchanged effective-configuration hash, **WHEN** reconciled, **THEN** no Application mutation occurs.
27. **GIVEN** concurrent Team work, **WHEN** reconciliation runs, **THEN** default concurrency is bounded at four workers; platform reconciliation is serialized and Team mutation waits for shared dependency changes.
28. **GIVEN** a missed event or restart, **WHEN** periodic resynchronization occurs, **THEN** drift is repaired idempotently.
29. **GIVEN** platform components, **WHEN** deployed, **THEN** explicit stages order foundations; networking, certificates, secrets and identity; shared services; Team foundations; and Team applications.
30. **GIVEN** a dependency is unhealthy, **WHEN** a dependent component is considered, **THEN** it remains blocked with precise status while independent components continue.
31. **GIVEN** an owned Application changes health without a spec change, **WHEN** observed, **THEN** owning CR readiness updates without regenerating its spec.
32. **GIVEN** the current generation and required Applications are healthy, **WHEN** status is evaluated, **THEN** `Ready=True`, `observedGeneration` is current, and standard conditions, last success time, and concise component results are present.
33. **GIVEN** status is stored, **WHEN** inspected, **THEN** it contains no secret data or full effective configuration.
34. **GIVEN** the emergency pause annotation, **WHEN** reconciled, **THEN** mutation stops and status reports `ReconciliationPaused`; neither CR exposes `spec.suspend`.
35. **GIVEN** `PlatformSettings.spec.version`, **WHEN** reconciled, **THEN** it resolves an immutable OCI platform bundle pinned by digest and all PlatformTeams inherit that version.
36. **GIVEN** an unsupported version or downgrade, **WHEN** reconciled, **THEN** existing Applications remain unchanged and status reports `UnsupportedVersion` or `DowngradeNotSupported`.
37. **GIVEN** a failed upgrade, **WHEN** reconciliation ends, **THEN** last-known-good Applications remain and the platform reports degradation.
38. **GIVEN** a PlatformTeam, **WHEN** its primary namespace is derived, **THEN** it is `team-` and is not configurable in `v1alpha1`.
39. **GIVEN** `v1alpha1`, **WHEN** its schema is inspected, **THEN** additional namespace assignment is absent.
40. **GIVEN** independently owned resources reference a deleting PlatformTeam, **WHEN** finalization runs, **THEN** deletion is blocked and remaining kinds and counts are reported.
41. **GIVEN** a deletable PlatformTeam, **WHEN** finalized, **THEN** managed Applications and ephemeral resources are removed while namespaces, volumes, repositories, and external data remain.
42. **GIVEN** Teams still exist, **WHEN** PlatformSettings deletion is requested, **THEN** deletion is blocked; ordinary CR deletion never performs irreversible decommissioning.
43. **GIVEN** supported legacy values, **WHEN** the migration CLI runs, **THEN** it emits deterministic, schema-valid CR manifests using shared Go API types and validation.
44. **GIVEN** legacy `_rawValues` or unsupported settings, **WHEN** migration runs, **THEN** it fails before cutover and reports every actionable path without dropping data.
45. **GIVEN** shadow mode, **WHEN** CRs reconcile, **THEN** intended output is comparable but no cluster mutation occurs.
46. **GIVEN** cutover, **WHEN** the TypeScript reconciler is active or shadow comparison has not passed, **THEN** Go mutation mode is not enabled.
47. **GIVEN** replacement reconcilers for excluded Team entities are incomplete, **WHEN** retirement is evaluated, **THEN** the TypeScript operator is not fully retired.
48. **GIVEN** any Go operator mode, **WHEN** runtime dependencies are inspected, **THEN** it never invokes or requires the TypeScript CLI or Helmfile.
49. **GIVEN** a fresh installation, **WHEN** bootstrap runs, **THEN** Helm installs CRDs, the single-replica operator, Argo CD, Sealed Secrets, and certificate plumbing before normal reconciliation.
50. **GIVEN** recovery, **WHEN** encrypted manifests return, **THEN** the Sealed Secrets key is restored before Argo CD applies them.
51. **GIVEN** unhealthy bootstrap dependencies, **WHEN** PlatformSettings reconciles, **THEN** deployment remains blocked with precise dependency status.
52. **GIVEN** the deployed operator, **WHEN** inspected, **THEN** it runs as one replica without leader election.

## Non-Functional Requirements

- **Security:** Never persist or log secret material; restrict Secret access to fixed platform or derived Team namespaces.
- **Least privilege:** Limit operator RBAC to resources and namespaces required for reconciliation, status, ownership, and watched dependencies.
- **Reliability:** Reconciliation is idempotent, retry-safe, cancellation-aware, and recoverable after restart or missed events.
- **Availability:** Operator downtime may pause reconciliation but must not interrupt workloads already managed by Argo CD.
- **Performance:** Team changes never trigger a full platform apply; dependency-aware enqueueing and hashes avoid unnecessary work.
- **Scalability:** Bound Team concurrency, avoid serializing unrelated Teams, and prevent unbounded status history.
- **Consistency:** Serialize platform mutations and pause dependent Team mutations while shared settings change.
- **Observability:** Expose stable condition reasons, actionable messages, Kubernetes events, and metrics for duration, retries, and failures.
- **Auditability:** Git remains the durable record of user intent without generated-output churn.
- **Compatibility:** Operator releases declare supported platform versions; release artifacts are immutable and digest-pinned.
- **Schema quality:** CRDs are structural, reject unknown fields, and validate local invariants without depending on operator availability.
- **Data safety:** Invalid settings, missing dependencies, unsupported versions, and failed upgrades preserve last-known-good Applications.
- **Ownership safety:** Surface Server-Side Apply conflicts and unapproved collisions instead of force-overwriting them.
- **Determinism:** Identical inputs produce identical effective hashes, desired Applications, and migration output.
- **Maintainability:** Go API types are the sole long-term schema authority and are shared by controllers and migration.
- **Testability:** Core behavior is observable through Kubernetes or migration CLI interfaces without private implementation assertions.

## Implementation Decisions

- Build `apl-operator` in a separate Go repository with controller-runtime, Kubernetes APIs, and Helm SDK.
- Define cluster-scoped `PlatformSettings` and `PlatformTeam` under `akamai.io/v1alpha1`.
- Enforce the PlatformSettings singleton through the fixed name `platform` and CRD validation.
- Make Go API types authoritative and retain the legacy schema only for migration.
- Limit PlatformTeam to Team settings and typed Team app settings or resource overrides.
- Use explicit typed override fields and preserve unset versus explicit zero values.
- Keep Git as durable intent, Argo CD as deployer, and live CRs as controller input.
- Resolve Secret references from fixed namespaces and watch them for retries.
- Reconcile inline Helm values into owned Applications using Server-Side Apply.
- Use explicit ownership labels, separate field managers, a dependency graph, and staged reconciliation.
- Select immutable, digest-pinned OCI bundles through PlatformSettings version.
- Handle state-dependent invalidity during reconciliation instead of a required validating webhook.
- Run one replica without leader election; serialize platform work and bound Team concurrency.
- Provide a separate migration CLI and non-mutating shadow mode; use no temporary adapters.
- Protect deletion with finalizers and separate irreversible decommission workflows.
- Bootstrap the minimum control plane through Helm.

## Testing Decisions

- Use one primary Kubernetes-facing acceptance seam: submit CRs, Secrets, and Argo CD Application state through an `envtest` control plane and assert only visible Applications, conditions, events, finalization, and ownership behavior.
- Use one separate black-box seam for the migration CLI: provide values-repository fixtures and inspect process results, diagnostics, and emitted manifests.
- Do not test private reconciler helpers, queue internals, or exact call sequences.
- Exercise structural and CEL validation through the API server.
- Exercise inheritance, dependencies, hashes, deterministic rendering, Secret recovery, ownership conflicts, lifecycle, versions, staging, health propagation, and shadow mode through externally visible behavior.
- Use real Server-Side Apply behavior in the test control plane.
- Add ephemeral-cluster integration tests only where envtest is insufficient: Argo CD sync and health, real Helm rendering, Sealed Secrets recovery order, and bootstrap.
- Use existing command, migration, Application-generation, Sealed Secrets, and Kubernetes interaction tests as behavioral prior art while moving the Go implementation to the higher Kubernetes seam.

## Out of Scope

- APIs or controllers for Team services, workloads, builds, code repositories, and network policies.
- Users or Team membership lists in either CRD.
- Additional Team namespace assignment in `v1alpha1`.
- Arbitrary chart overrides, `_rawValues`, generic patches, or unknown fields.
- Writing generated configuration or status to Git, or synchronizing live specs back to Git.
- Direct workload deployment by the operator.
- Runtime adapters to TypeScript CLI or Helmfile.
- Automatic downgrade or generic rollback.
- Irreversible deletion from ordinary CR deletion.
- Multiple operator replicas or leader election.
- Final TypeScript operator retirement before excluded entities have replacement reconcilers.

## Further Notes

- Canonical domain terms are PlatformSettings, Team, PlatformTeam, Primary Team Namespace, and Namespace Assignment.
- A PlatformTeam may exist temporarily without PlatformSettings due to Git apply ordering, but cannot become operational.
- Team-relevant platform dependencies include domain, gateway class, certificate issuer, identity, selected shared-app enablement, service-mesh revision, Git source, image access, and object storage. The dependency map must be explicit and tested.
- Migration is intentionally strict: unsupported settings block cutover rather than being silently retained or discarded.
- This PRD implements the accepted architectural decision for platform configuration through PlatformSettings and PlatformTeam CRDs.

No implementation files or tests are named; the work targets a new Go apl-operator repository. Start by mapping the shared Go API types, CRDs, controllers, and migration CLI described in the acceptance criteria. Done requires the typed resources, reconciliation, migration, ownership, dependency, status, security, and bootstrap behaviors to satisfy the specified criteria.

Written by the indexing model from the issue text.