Proposal: Pre-validate haproxy.cfg before Secret update using a short-lived Job

Hi !

To improve the safety and reliability of the Operator, I'm currently considering implementing the mechanism described below.
Would you be open to accepting a pull request for this once I have it ready?

If this approach doesn't align with the project's design direction, please feel free to reject the idea—no worries at all!

Configuration Validation Before Secret Update

Before the operator updates the runtime configuration Secret (<instance>-haproxy-config), it validates the generated haproxy.cfg using a short-lived Kubernetes Job:

1. Build the generated haproxy.cfg and all referenced certificate/config files.

2. Create a temporary validation Secret.

3. Start a short-lived Job using the same image as the target HAProxy instance (spec.image, fallback haproxy:latest) and run:

   haproxy -c -f /usr/local/etc/haproxy/haproxy.cfg

4. If validation succeeds, update the runtime configuration Secret.

5. If validation fails, do not update the runtime configuration Secret, and set the Error status on the Instance and related configuration resources.

6. Clean up temporary validation resources (Job and Secret) automatically (via TTL) and explicitly by the operator.

Looking forward to hearing your thoughts!

Start by tracing the operator reconciliation path that generates haproxy.cfg and updates the runtime configuration Secret, then review how Instance and related configuration resources receive Error status. Define the validation Job and temporary Secret lifecycle, including success, failure, TTL, and explicit cleanup; done means invalid configurations leave the runtime Secret unchanged while valid ones update it safely.