Improve tracing / reconciliation when clusterOperation.reconciliationPaused is set to true

When clusterOperation.reconciliationPaused is set to true, the operator still runs through reconciliation without applying anything. This leads to errors in the operator logs when required dependencies are not applied and missing (e.g. retrieve a nodeport from a service etc.)

2024-02-13T09:18:40.770253Z ERROR stackable_operator::logging::controller: Failed to reconcile objec │
│ t controller.name="opacluster.opa.stackable.tech" error=reconciler for object OpaCluster.v1alpha1.op │
│ a.stackable.tech/opa.platform failed error.sources=[failed to apply role Service, Kubernetes reporte │
│ d error: ApiError: services "opa" not found: NotFound (ErrorResponse { status: "Failure", message: " │
│ services \"opa\" not found", reason: "NotFound", code: 404 }), ApiError: services "opa" not found: N │
│ otFound (ErrorResponse { status: "Failure", message: "services \"opa\" not found", reason: "NotFound │
│ ", code: 404 }), services "opa" not found: NotFound]  

This shadows the real problem (in this case it was hard to see that the operator is actually not reconciling that cluster) and the errors is logged constantly in the operator and the CR events.

We need to improve the operator logging in this case to "ignore" errors and simply log that this cluster is currently not reconciled.

This may affect other operators as well.

Start by locating the reconciliation path that handles clusterOperation.reconciliationPaused and review how errors are logged and emitted as CR events. Done means a paused cluster does not produce repeated dependency errors and instead clearly logs that reconciliation is paused; check whether the behavior applies to other operators as suggested.