Chart: Helm Refurbish

### Body

As part of our ongoing Helm chart refurbish work in Apache Airflow, we will follow a structured approach to ensure stability for existing users while preparing for a clean, well-defined next major release.

https://cwiki.apache.org/confluence/display/AIRFLOW/Helm+Refurbish
https://cwiki.apache.org/confluence/display/AIRFLOW/Direction+and+Plan+for+Chart+2.0?src=contextnavpagetreemode

Single tracker for the Helm chart 2.0 refurbishment. Each section below is a category of work; the items inside are the concrete action items. Tick as they land. Larger items may spawn dedicated PRs and issues — link them from the relevant action item rather than removing the checkbox.

Contributor guide: Developing the Helm Chart (coming in #66659). Canonical chart-vs-Kustomize criteria: `chart/kustomize-overlays/CONTRIBUTING.rst`. The chart never removes a component without a working overlay already in place.

---

#### Deprecate and remove legacy keys

Deprecation warning ships on 1.2x for at least one minor before removal on main.

- [ ] Deprecate `apiSecretKeySecretName` on 1.2x; remove in 2.0
- [ ] Deprecate `apiServer.apiServerConfig` on 1.2x; remove in 2.0
- [x] Remove misplaced ingress keys: `registry.secretNames`, `securityContext`, `ingress.apiServer.host`, `ingress.apiServer.tls.enabled`, `ingress.apiServer.tls.secretName`, `ingress.enabled`
- [x] Stage all webserver-related deprecations on main
- [ ] Audit remaining Airflow-below-3.0 compatibility keys; deprecate then remove

#### Relocate parameters to their owning sections

No functional change — parameters work today but live under the wrong parent. Canonical name changes.

- [ ] Move Redis parameters under the celery section
- [ ] Move Jobs parameters under the kubernetes section
- [ ] Consolidate `data.celery.*` under a celery-specific heading
- [ ] Align `podDisruptionBudget` shape with `pod_template_file` (Kubernetes executor pattern)
- [ ] Audit remaining parameters under `kubernetes.*` that belong elsewhere; relocate

#### Restructure persistence model around bundles

Persistence becomes a bundle configuration. Log and Dag folders split into two distinct types: Log and Bundles.

- [ ] Split persistence configuration into Log and Bundles types
- [ ] Implement single `DagBundle` per deployment
- [ ] Support multiple `DagProcessor` instances per bundle
- [ ] Design multi-team support against the bundle shape (not retrofitted onto the current model)

#### Migrate components to Kustomize overlays

Components delivered as overlays rather than chart-native templates. Each item may spawn its own PR/issue.

- [ ] Kerberos overlay (sidecar-injection pattern)
- [ ] Document `GitDagBundle` migration; remove gitSync from chart afterwards (overlay only if community demand persists)
- [ ] Elasticsearch / OpenSearch overlay
- [ ] PostgreSQL StatefulSet overlay (chart stops shipping a database)
- [ ] StatsD overlay (OpenTelemetry default in chart)
- [ ] HPA / KEDA overlay (standalone-addition pattern)
- [ ] General telemetry / monitoring overlays beyond the OpenTelemetry default

#### Decouple `airflow.cfg` from the chart

All configuration expressed through environment variables. Chart keeps a minimal cfg surface; complex per-component config moves to overlays.

- [ ] Express all chart configuration via environment variables
- [ ] Reduce chart cfg surface to basics
- [ ] Move complex per-component configuration to overlays
- [ ] Investigate per-component cfg (separate cfg per Airflow component) for the major bump

#### Consolidate duplicate configuration paths

A given setting is reachable from one section. A given image is defined in one place.

- [ ] Normalise Images section so each image is defined in one place rather than two
- [ ] Reshuffle image-related parameters into the components they belong to
- [ ] Audit remaining settings reachable from multiple sections; consolidate

#### Tighten chart defaults to a least-privilege baseline

- [ ] Enforce security context at the component and container level
- [ ] Apply least-privilege defaults consistently to roles and role bindings
- [ ] Provide sensible `serviceAccount` defaults; allow finer shaping via overlays
- [ ] Readiness / liveness probes cover the common case (~80%) in the chart; remaining shapes via overlays

#### Open scoping investigations

Spikes that inform later decisions and do not directly produce chart changes. Outcomes feed back into the relevant section above.

- [ ] PgBouncer for Airflow v3 and higher: load test or equivalent verification to decide whether it still adds value
- [ ] Migration image and `waitForMigrations`: revisit against newer Kubernetes features to decide whether the two-image arrangement and explicit wait step are still required
- [ ] Multi-image solution placement (chart vs overlay): pending component-isolation work

### Committer

- [x] I acknowledge that I am a maintainer/committer of the Apache Airflow project.