Automatic periodic CA (key) rollover

Is your feature request related to a problem? Please describe.

At the moment Icinga 2 generates a cluster-wide CA once for a lifetime.
Actually for 14.99 years which, technically speaking, becomes the lifetime of a cluster.

• #9890 corrects the latter, but still re-uses the public key.

Not a security disaster by itself, but we can do better.

Describe the solution you'd like

Roots. The v2.X.0+ master periodically creates a new CA named Icinga CA %Y, adds it to local trust bundle. For the obvious reason to distrust the existing CA one nice day. The latter excludes cross-signing the existing one. So only cross-signing the new one is an option. Theoretically this would allow issuing with the new CA immediately. And the current certificate distribution mechanism even seems to allow deploying chains with intermediate CAs. But only deploying, not recognising CSRs. 👎 Not w/o #9795. So the master can't just issue X using the new CA without knowing whether all cluster levels between itself and X have at least #9795. Or v2.{X-1}.0+ must have #9795 and is a requirement for all satellites under a v2.X.0+ master. This would even work with current agents as they'd just have to recognise the satellites as such, not their CSRs which #9795 is about. Cross-signing even seems the only option with smooth transition. Otherwise the new CA would be pure spare until the master knows for sure that all nodes also trust it. How? Via another layer of complexity in cluster communication and synchronisation? 👎

Leaves. As the leaf validity is async. with the root validity, there may be old leaves valid for 10y+ not needing renewal. Once the master wants to retire their CA, they do need. I.e. the master forcibly renews certs not issued with the new CA. Another reason to update satellites.

</walloftext>

TL;DR

• v2.X.0+, satellite
  • #9795 (recognise CSRs across intermediate CAs)
  • Deploy the whole trust bundle down, not just "the only" root CA (optional?)
  • Forcibly renew up-to-date leaves not signed with newest trusted CA
• v2.{X+1}.0+, master
  • Periodically add a new CA named Icinga CA %Y to trust bundle
  • Cross-sign it with existing one
  • Use resulting cert for issuing leaves (deploy the whole chain down)
  • Retire old CA by oneself one nice day (or let not-valid-after time do its job)
  • Reduce new CAs' validity (optional)

Additional context

The only question is:
Do customer get a problem with the API if the Icinga CA suddenly changes?
Can we expect them to regularly update their handcrafted curl trust stores?

Start by reading the related issues #9795 and #9890, then trace the existing CA generation, trust-bundle deployment, CSR recognition, and leaf-renewal paths. Clarify the compatibility requirements for masters, satellites, agents, and handcrafted API trust stores. Done means the rollover, cross-signing, distribution, forced renewal, and retirement behavior are specified and covered across supported cluster versions.