Scorecard shows warnings from k8s API response when Operator making requests to deprecated REST APIs

Feature Request

Describe the problem you need a feature to resolve.

I'd like to see in the default Scorecard tests results about the deprecation/removal of APIs in the next versions.

Goal

The goal is to check if Scorecard could gathering the info that is already provided by Kubenertes API when the operator is running on the cluster does request using deprecated APIs.

No Goal

• Implement any logic which will check if a deprecated API is or is not used by the operator. It is already done by the k8s api
• Implement any logic that will lint the files/manifests. Statics checks on the manifests are already addressed via the OperatorHub validator on the bundle validate command.
• Implement a new feature that allows performing tests on the Operator running on the cluster. That is addressed by Scorecard already.

Motivation

The Scorecard is the SDK feature that is capable of running tests on the cluster. In the same way that is recommended to perform regression tests on the projects and then, check its events/metrics to see if any warnings about deprecated APIs were raised it would be gathered by Scorecard default tests and appended to its results.

Use Case

I am as operator author, I would like to be able to gather the warnings raised by the k8s API when my operator is running and tested by Scorecard on the cluster then, I would be able to be easily informed beforehand that my operator is using/doing requests to deprecated APIs

Describe the solution you'd like.

Implementation in the Scorecard checks to looking for the raised events/metrics in the K8S API to gathering its WARNINGS and return as a result of its tests. More info: https://kubernetes.io/blog/2020/09/03/warnings/#deprecation-warnings. E.g:

1. Run Scorecard tests to trigger the reconcile by for example applying its CRs on the cluster
2. Then, gathering the metrics raised by K8s API and append its result in the Scorecard results. (e.g We can check the warnings by looking at the events e.g kubectl get events --field-selector="reason=AppliedWithWarnings" --all-namespaces) :

OR

kubectl get --raw /metrics | prom2json | jq '
.[] | select(.name=="apiserver_requested_deprecated_apis").metrics[].labels
'

3. Appending the WARNINGs to the Scorecard results

Aditional Context:

OCP introduce Prometheus alerts using k8s metrics
Two alerts have been introduced with OpenShift 4.8

• APIRemovedInNextReleaseInUse - for APIs that will be removed in the next release.
• APIRemovedInNextEUSReleaseInUse - for APIs that will be removed in the next EUS release.

More information on alerts, how to retrieve them or how to get notified is available in OpenShift documentation.

/language go
/language ansible
/language helm

Start by reading the Scorecard test implementation and its result-reporting path, then review how the Kubernetes API exposes deprecation warnings through events and metrics. Compare the proposed event and Prometheus approaches, and define which source, permissions, scope, and result format would be supported before implementation; done means warnings from Scorecard-triggered requests are reliably included in test results.