Streaming aggregation config files as CRDs in VM operator for K8s

Is your feature request related to a problem? Please describe

Having CRDs to scrape different services is great, as we can have on each microservice repository a folder, for instance, /monitoring, where developers can have their vmservicescrapes CRDs that are later deployed together with the service in K8s and read by vmagent, hence start the scrape of each new service.

The problem is that, if the developer wants to reduce cardinality of the metrics scraped with some Streaming Aggregation config, right now the only way to do it is by changing the vmagent deploy to add in a single file all the streaming aggregation configs with an arg, like

-remoteWrite.streamAggr.config=/etc/vm/stream-aggr-configs/stream-aggr-configs.config

This stops the developers to control their own metrics and leaves the team in charge of the vmagent deploy to maintain and "gatekeep" all the configuration regarding all the services in a company in a single (and possibly massive) file.

Describe the solution you'd like

The same way we have CRDs for vmnodescrapes vmpodscrapes and vmservicescrapes, we should be able to deploy vmstreamaggrconfigs.

vmagent can then read the CRDs and apply the configs the same way is doing with the scrape configs.

Describe alternatives you've considered

We created a separate repository where the developers can drop files with their stream aggregation config files, then the CI merges all the files into a single one that is deployed as a ConfigMap in Kubernetes and mounted in a dir with an specific filename where, by default, vmagent is reading with -remoteWrite.streamAggr.config=/etc/vm/stream-aggr-configs/stream-aggr-configs.config.

This doesn't really solve the issue but at least mitigates it so developers doesn't have to do changes in the vm k8s operator code.

We still have to gatekeep the code and will have a massive file sooner than later.

Additional information

An "easy" solution will be applying a similar approach that VM already have with servicescrapes

Start by tracing the existing vmservicescrapes CRD approach in the VM Kubernetes operator and compare how vmagent reads scrape configurations. Define the corresponding vmstreamaggrconfigs behavior and verify that deployed CRDs are accepted and their streaming aggregation settings are applied by vmagent.