Make AppProxy coordinator stateless by removing its own DB
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 15h 13m
- Merged PRs (30d)
- 368
Description
Motivation
The AppProxy coordinator keeps its own PostgreSQL database while Traefik reads the same routing state from etcd. For circuits the duplication is total: the whole circuit record is serialized into the etcd middleware payload (models/circuit.py:469-471) on top of the router and service subtrees. Keeping the two stores in sync is what surfaced as BA-5499 and again as BA-7121, where a stale in-memory snapshot overwrites newer etcd state. Neither the per-circuit asyncio lock nor the 30s reconcile loop structurally prevents it, because both leave the DB read outside the critical section and the coordinator is designed to run as multiple instances.
Objective
Remove every DB access from the coordinator and make it a stateless applier. Ownership splits into the manager DB (source of truth) and etcd (shared state). Components integrate through etcd and Valkey only; the coordinator never queries the manager.
State relocation
- endpoints (health check config): the manager already owns the original; the coordinator keeps only what it renders into etcd.
- circuits: etcd.
- workers and worker_app_filters: etcd with leases, replacing the 15s LOST-detection polling.
- tokens: JWT or Valkey TTL.
- occupied_slots: derived from the circuit count in etcd, so the counter and its drift reconciler disappear.
Convergence
Convergence responsibility moves from the coordinator to the manager. The coordinator-side reconciler is deleted, since a stateless applier has no desired state to reconcile against. The manager pushes declarative per-endpoint desired state at create and terminate points, and runs a low-frequency sweep to catch lost calls, orphaned keys, and etcd restores.
Design is tracked by BEP-1073. Implementation issues are defined after the BEP settles.
JIRA Issue: BA-7123
Contributor guide
Research direction
Start by reading BEP-1073 and the circuit serialization references in models/circuit.py:469-471. The implementation issues and completion criteria are explicitly deferred until the BEP settles, so there is no defined entry point or test target yet.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python
- Domain
- backend, databases, distributed-systems
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100