Support Cluster Metadata K8s Annotation
- Dominant language
- C#
- Stars
- 9.6k
- Forks
- 933
- Avg merge
- 12d 18h
- Merged PRs (30d)
- 2
Description
### What should we add or change to make your life better?
The ability to add Metadata to clusters generated by the K8s controller `YarpParser` in addition to the recently added Route Metadata (#1518).
### Why is this important to you?
We are using route/cluster metadata heavily in our project to reduce the amount of "static" configuration required. We use it within `ITransformProvider`/`IProxyConfigFilter` to build up preset configurations for both routes and clusters.
More importantly, I am also tracking backend clusters via `IClusterChangeListerner`. I look for clusters with specific Metadata to identify them as candidates for backends that support a swagger spec so that I can present a unified version hosted in my YARP gateway.
At the moment we are using the `ConfigurationConfigProvider` so I can just add the Metadata directly to the cluster in configuration. However, we hoped to switch over to a combination of the `ConfigurationConfigProvider` *and* the K8s provider once the changes from #1534/#979 are published. Our service/API routes/clusters would now be generated by the k8s provider, with our static/non-k8s setup remaining in config.
With #1518 the k8s parser now adds Route Metadata but does not add cluster Metadata. This leaves me with no way to to identify the generated cluster as a candidate in my change listener.
Since I currently must build the k8s controller myself, its simple enough to add this in, but it seems like something that should already be there.
I suppose there's the problem that multiple routes can (apparently?) end up with the same cluster, so you'd either overwrite the Metadata ([as with other cluster options](https://github.com/microsoft/reverse-proxy/blob/cc0d2ed0563ddd5d09c504e0fbdac69f1fa5c72a/src/Kubernetes.Controller/Converters/YarpParser.cs#L69-L74)) or need a strategy for merging them.
### Alternatives
Instead of introducing a new k8s annotation to split out route vs cluster metadata, the route Metadata could just be copied to the `ClusterConfig` as well.
If there were a way to retrieve all of the routes associated to a cluster, I could instead just loop over the routes and look for the Metadata *there* as part of my change listener processing.
Another option would be some way of allowing `IProxyConfigFilter` to make modifications to the cluster associated with the route in `ConfigureRouteAsync`. I could then just copy/append the annotations from route to cluster. This however probably has ordering concerns since `ConfigureClusterAsync` is called first. Plus I doubt you'd ever want to allow modifying anything about the cluster other than Metadata, and even that seems unique to my use case.
Contributor guide
Assessment
This issue has not been assessed yet.