Dstack-TEE / Dstack-TEE/dstack
gateway: concurrent ACME rotation is only best-effort serialized
- Lingua principale
- Rust
- Stelle
- 544
- Fork
- 96
- Merge medio
- 17h 57m
- PR unite (30g)
- 117
Descrizione
Follow-up to #935.
`RotateAcmeCredentials` is serialized across nodes by a TTL lock in WaveKV (`global/acme_rotation_lock`). WaveKV is last-writer-wins without compare-and-swap, so the lock is acquired by read-then-write: two nodes calling the RPC within a replication gap can both acquire it and rotate concurrently.
## Impact
If two rotations interleave, CAA records end up pinned to one node's new account while LWW keeps the other node's credential in KV. Since #935 the state is recoverable — the published credential wins LWW and one `SetCaa` run re-pins every domain to it — but issuance is broken until an operator notices and intervenes.
The constraint "rotate through one gateway at a time" is advisory only: the admin endpoint uses a shared bearer token with no per-method authorization, so nothing enforces it.
## Possible directions
- Add CAS (or a fenced-lock primitive) to WaveKV and make the rotation lock a real mutex.
- Route rotation to a designated leader node instead of accepting it on any gateway.
- At minimum: automatic post-rotation verification that the published credential's `accounturi` matches every domain's CAA, alerting on divergence (see the CAA reconciliation issue).
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Start by reading the RotateAcmeCredentials RPC, the WaveKV global/acme_rotation_lock behavior, and the follow-up context in #935. Compare the listed CAS or fenced-lock, leader-routing, and post-rotation verification directions with the gateway’s current behavior. Done should prevent concurrent rotations or reliably detect and recover from CAA and credential divergence.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- rust
- Ambito
- distributed-systems
- Tipo di issue
- Bug
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Stato di attività
- Tranquilla
- Chiarezza
- Da chiarire
- Idoneità per principianti
- 35/100