Reconciler with enabled "generationAwareEventProcessing" reconciles unchanged custom resource upon startup

Bug Report

What did you do?

Hello,

We've encountered the behavior where we're using the default Reconciler settings, which has generationAwareEventProcessing turned on as well as a default maxReconciliationInterval set to 10 hours. Our CR implements the ObservedGenerationAwareStatus interface, so it has the observedGeneration property present in the status and it's handled by the framework.

We're seeing that upon Operator startup a reconciliation is triggered, when there is an already created custom resource in the cluster which has been reconciled already and has metadata.generation equal to status.observedGenertion.

The problem in this scenario is that the Operator is being updated and has updated logic, triggered by a pipeline automation script update. An update to the CR is going to follow in the next pipeline step. However, the triggered reconciliation captures the current CR in the cluster before the updated one can be applied. That in some cases leads to breaking dependent resources spec and subsequent failure (e.g. old incompatible images read from the CR set to an updated Deployment dependent resources spec encoded in the Operator code).

What did you expect to see?

No reconciliation triggered on the CR, which has it's generation and observed generation matching properties.

Environment

Kubernetes cluster type:

Gardener K8s 1.33.5

java-operator-sdk version is 4.4.4

Java version 17

Possible Solution

Not sure if the behavior could be caused by the maxReconciliationInterval being not set after the update or just informer caches being empty.

No source file or test is named. Start by reproducing operator startup with generationAwareEventProcessing enabled and an existing custom resource whose metadata.generation equals status.observedGeneration; trace startup reconciliation and informer-cache handling. Done means unchanged resources are not reconciled on startup, with a regression test covering this case.