Simplify config-CRD reference tracking (indexers, framework dedup, on-demand deletion)
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 2.2k
- Forks
- 300
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 184
Description
Summary
The six config-CRD controllers (MCPOIDCConfig, MCPAuthzConfig, MCPExternalAuthConfig, MCPTelemetryConfig, MCPWebhookConfig, MCPToolConfig) each track which workloads reference them in Status.ReferencingWorkloads. The way this is maintained is more complex and less efficient than it needs to be, and the stored list itself is arguably not worth keeping. This issue proposes three incremental, mostly-independent improvements — from a safe drop-in win to a longer-term simplification.
Related: refactor work started in #5599 (sharing the stale-ref scan) — this issue reframes that as "the scan can largely be deleted, not deduplicated."
TL;DR
- Use field indexers instead of list-all-and-filter when computing referencing workloads (cheaper on every reconcile, idiomatic, already used for
spec.groupRef). - Delete the hand-rolled "stale ref" scan — controller-runtime already wakes the previously-referenced config for free.
- (Longer term) Drop
Status.ReferencingWorkloads— it's effectively only there for a printer column, it's expensive to maintain at scale, and it doesn't help deletion protection (which re-queries live anyway).
What's in place today
- Each config stores
Status.ReferencingWorkloads []{Kind,Name}+ aReferenceCount(surfaced as thekubectl"References" printer column). - Each controller watches the referring workload types (MCPServer / VirtualMCPServer / MCPRemoteProxy) via
EnqueueRequestsFromMapFunc. - The watch map function does two things: enqueue the config the workload references now, and a "stale ref" scan — list all configs of that type and check each one's
Status.ReferencingWorkloadsto find configs that still list a workload that no longer references them, so they reconcile and prune the stale entry. - On reconcile,
findReferencingWorkloadsrebuilds the list by listing every workload in the namespace and filtering in memory. - Deletion protection uses a finalizer:
handleDeletioncallsfindReferencingWorkloadslive and blocks while the count > 0.
The redundancy / inefficiency
- The stale-ref scan is redundant. controller-runtime's
EnqueueRequestsFromMapFuncruns the map function on both the old and new object on every update (verified in our version, v0.23.3 —pkg/handler/enqueue_mapped.goenqueuesObjectOldandObjectNew). So when a workload switches its ref from A→B (or drops it, or is deleted), the framework already enqueues A via the old object. The hand-written scan is re-solving a problem the framework solves — and it's copy-pasted across all six controllers. - The rebuild is O(all workloads).
findReferencingWorkloadspulls every workload in the namespace from the cache and filters in memory, on every reconcile. At hundreds of workloads this is wasteful CPU/allocations on the hot path. - The stored list earns almost nothing. It is read by nothing except the printer column; the deletion decision recomputes live and does not trust it.
What we get from the framework for free
EnqueueRequestsFromMapFunc already maps old + new on updates (and the object on create/delete). So the map function only needs to return "the config this workload references", and the config a workload just left is woken automatically — no status scan required. Pair it with a predicate so it only fires on real ref changes (avoids status churn). Missed-event edge cases (e.g. a delete while the controller was down) are healed by the level-triggered reconcile + periodic resync, same as today.
Better way: field indexers for the rebuild
Register a field index on each workload's ref field and query it directly:
mgr.GetFieldIndexer().IndexField(ctx, &MCPServer{}, "spec.oidcConfigRef", extractRefName)
// ...
r.List(ctx, &servers, client.InNamespace(ns), client.MatchingFields{"spec.oidcConfigRef": cfg.Name})
This returns only the matching workloads instead of scanning all of them. It's the standard controller-runtime/kubebuilder pattern and is already used in this repo for spec.groupRef (setupGroupRefFieldIndexes, mcpserverentry_controller). Same output, much cheaper at scale, and it benefits both the reconcile rebuild and the deletion check.
Longer term: compute references on-demand, drop the stored list
Status.ReferencingWorkloads only exists to back the printer column. It does not protect deletion — handleDeletion already re-runs the lookup live before removing the finalizer, so the stored list is pure display overhead. Maintaining it means watching every workload type and rewriting config status on each change, which scales poorly (hundreds of workloads → constant status writes for a count nobody reads programmatically).
The conventional pattern (e.g. Kubernetes PVC protection: pkg/controller/volume/pvcprotection) is finalizer + on-demand check at deletion time, no stored list. We could drop ReferencingWorkloads (and the watches/index that feed it), keep just the finalizer + live check, and either drop the "References" column or keep a lightweight count if the UX is worth it. This removes the entire watch + rebuild + store + scan apparatus.
Efficiency / outcome
- Reconcile rebuild: O(all workloads) in-memory scan → O(matching workloads) indexed lookup.
- Watch path: delete the per-controller stale-ref scan (and its shared helper + getters) — net code removal, not addition.
- At scale (hundreds of workloads/configs): far fewer wasted reconciles and status writes; if we go all the way to on-demand, we stop watching workloads purely to maintain a display count.
- No behavior change for steps 1–2:
Status.ReferencingWorkloads, the count column, and finalizer-based deletion protection all stay identical — just computed more efficiently.
Suggested order
- Field indexer for
findReferencingWorkloads(safe, drop-in, no behavior change). - Delete the stale-ref scan; lean on framework old+new enqueue + a predicate (verify with the operator integration suite).
- Discuss whether to drop
Status.ReferencingWorkloadsand move fully to on-demand deletion checks.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the six config-CRD controllers and their findReferencingWorkloads implementations, then compare setupGroupRefFieldIndexes and mcpserverentry_controller for the existing indexing pattern. Run the operator integration suite while checking the old/new enqueue behavior and ref-change predicate. Done means indexed lookups and removal of the redundant stale-ref scan, with behavior preserved; dropping stored status remains a separate longer-term decision.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- backend, devops, infrastructure
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100