HarperFast / HarperFast/harper

Certificate renewal can orphan a previous hdb_certificate row, leaving a stale cert bound to a rotated key

Open
#2,160 0 comments 0 reactions 1 assignee Claimed by @DavidCockerill View on GitHub
area:security bug
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

## Summary

`loadCertificates()` keys each `hdb_certificate` row by:

```js
certCn = (!ca && config.name) || getPrimaryHostName(x509Cert);

export function getPrimaryHostName(cert) {
const commonName = cert.subject?.match(/CN=(.*)/)?.[1];
if (commonName) return commonName;
return hostnamesFromCert(cert)[0]; // first SAN
}
```

If the input to that name ever changes — a `name` is added to the TLS config, or the certificate's CN/first-SAN changes between issuances — the next load writes a row under the **new** name and leaves the previous row in place. Nothing removes or updates it.

The orphan keeps its `private_key_name`, typically the same key file as the live row. When that key is next rotated, the orphan holds a certificate that no longer matches the key it points at.

## Why the orphan still breaks TLS

Route/SNI selection is built from each certificate's SANs, not from the row name:

```js
let hostnames = cert.hostnames ?? hostnamesFromCert(certParsed);
```

So an orphan still claims every hostname in its SANs. When a request arrives for one of those names, the orphan can be selected, `tls.createSecureContext` is called with a mismatched pair, and it throws:

```
Error applying TLS for
error:05800074:x509 certificate routines::key values mismatch
at tls.createSecureContext (core/security/keys.js)
at SNICallback.initialize
at createWebSocket (replication/replicationConnection.js)
at replicateOperation (replication/replicator.js)
code: 'ERR_OSSL_X509_KEY_VALUES_MISMATCH'
```

Observed on a two-node cluster where this broke **outbound replication** — the node could not build a TLS context for its replication socket, so `deploy_component` failed to replicate, which triggered a restart, which wedged. Only one of four rows was bad; the live row matched its key perfectly, so per-node hostnames kept working and the failure looked intermittent and cert-unrelated.

Restarts and version upgrades do **not** clear it — both re-read the same rows from disk.

## Reproduction shape

1. Load a certificate with no `name` in the TLS config and no CN → row is keyed by first SAN, e.g. ``
2. Later, add `name: ` to the TLS config → next load writes a second row keyed ``
3. Rotate the private key that both rows reference (a routine LE renewal)
4. The `` row refreshes; the `` row does not, and now mismatches
5. Any TLS handshake selecting the orphan for one of its SANs fails with `ERR_OSSL_X509_KEY_VALUES_MISMATCH`

## Detection

On any instance, `list_certificates` and look for **two `server`/`operations-api` certificates sharing the same `private_key_name` with different `valid_to` dates**. The older one is the orphan.

## Proposed fix

When `loadCertificates()` writes a row, remove or refresh superseded rows that reference the same `private_key_name` and whose SAN set is covered by the new certificate. That makes the naming change self-migrating rather than leaving a row that is guaranteed to break at the next key rotation.

Failing that, a startup reconciliation that drops non-authority rows whose certificate does not match the key they name would be a safe backstop — such a row cannot serve a valid handshake in any case.

## Workaround

`remove_certificate name=`. Note this deletes the private key file if the removed row is the **only** row referencing it (`certificate.ts` `removeCertificate`), so confirm at least one other row shares the `private_key_name` first.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.