autoTls: calculation for secure maximum certificate lifetimes so that all pods that are deployed trust every other deployed pod is too loose

Affected Stackable version

26.7

Current and expected behavior

When using the autoTls backend of the secret operator, the operator automatically issues new CA certificates as old ones expire and ensures that all CA certificates are rolled out to all pods of a cluster, so that everybody trusts everybody.

We have safe_max_cert_lifetime which calculates the maximum "safe" duration of a generated certificate that ensures a pod can never run too long. Too long in this case means: it is so old, that a new CA certificate has been generated after it was started, so it doesn't know this CA and other pods in the cluster are already using certificates signed with this ca -> trust is broken.

The current implementation of safe_max_cert_lifetime is: (caCertificateLifetime - caCertificateRetirementDuration) / 4 which I think is too loose.

I'll outline an example below that shows how this can break. Please please please: Someone actually double check everything here! I have done the calculations myself and it looks to be possible to me, but I'd very much like a second opinion on this!

I have pushed the test that surfaced this here: https://github.com/stackabletech/secret-operator/compare/main...feat/cert-lifetime-tests

I am not saying we should merge that, and even if we decide to merge it, it'll need cleanup, which I am happy to do. But I didn't want to spend the time on this before we decide we want this.

Example

Config

• CA lifetime = 100 days
• Retirement duration = 30 days
• Rotation fraction = 2

Our code accepts this — the only rule it enforces is that retirement is shorter than the CA lifetime.

How the code turns that config into numbers

• Active lifetime — a CA counts as active for its lifetime minus the retirement duration: 100 − 30 = 70 days.
• Rotation — a fresh CA is created once the newest one is within active ÷ rotation fraction = 70 ÷ 2 = 35 days of expiring. So new CAs are born 65 days apart (100 − 35).
• Longest leaf certificate handed out — active ÷ rotation fraction ÷ 2 = 70 ÷ 2 ÷ 2 = 17.5 days.
• When a CA can still be used — a CA can sign new certificates and anchor trust only up to its retirement point, expiry − retirement, not all the way to expiry.

The CAs
Day 0 is the creation day of CA1!!

CA	created	expires	usable until (expiry − 30)
CA₀ (old)	day −65	day 35	day 5
CA₁ (new)	day 0	day 100	day 70

Pod Timeline

Pod A mounts on day −12.5 — the latest it still fits under CA₀ (CA₀ is usable until day 5, and a 17.5-day leaf from day −12.5 ends exactly at day 5).

• signed by CA₀, trusts {CA₀} (CA₁ doesn't exist yet), alive until day 5.

Pod B mounts on day 0.

• CA₀ can't sign it — a 17.5-day leaf would run to day 17.5, but CA₀ is only usable until day 5 — so it's signed by CA₁.
• signed by CA₁, trusts {CA₀, CA₁}, alive until day 17.5.

Problem

Pod A:  [−12.5 ─────────── 5]           signed by CA₀   trusts {CA₀}
Pod B:            [0 ──────────── 17.5]  signed by CA₁   trusts {CA₀, CA₁}
                   └───────┘
				   overlap: 
                   days 0–5

On days 0 to 5 both pods are alive, but Pod A never learned about CA₁, so it can't verify Pod B's certificate. mTLS between the two silently fails.

Possible solution

The proposed fix is to change safe_max_cert_lifetime to (caCertificateLifetime - 3 * caCertificateRetirementDuration) / 4 which at least gets the test to pass.. not sure if this is my final suggestion though.

Additional context

No response

Environment

No response

Would you like to work on fixing this bug?

None

Start by reviewing the safe_max_cert_lifetime calculation in the secret operator and the certificate-lifetime tests linked in the issue's comparison. Verify the CA rotation and pod timeline assumptions, then add or update regression coverage so the chosen lifetime prevents overlapping pods from losing mutual trust.