Rate limiting and circuit breaker for transaction submission
- Dominant language
- Go
- Stars
- 4
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
Add backpressure mechanisms to the transaction submission path to handle DA layer degradation gracefully.
Parent: #4
## Context
celestia-node has zero rate limiting or backpressure on submission — just a mutex serializing broadcasts. Under load or during network issues, this leads to unbounded retries and resource exhaustion.
## Requirements
### Concurrency limiter
- Semaphore bounding max in-flight transactions (configurable, default: 4)
- Callers block when limit is reached
- Works with multi-account (#5): limit is global across all accounts
### Exponential backoff
- On repeated submission failures, increase delay between attempts
- Base: 1s, max: 30s, jitter: +/- 25%
- Reset backoff on successful submission
### Circuit breaker
- Track failure rate over a sliding window (e.g., last 20 submissions)
- States: closed (normal) → open (failing, reject immediately) → half-open (probe)
- Open threshold: >50% failure rate over window
- Half-open: allow 1 submission, if success → closed, if fail → open
- When open: return immediate error with "circuit open" context, don't waste gas
### Metrics (#7)
- `apex_submission_inflight` (gauge)
- `apex_submission_circuit_state` (gauge — 0=closed, 1=half-open, 2=open)
- `apex_submission_backoff_seconds` (gauge)
```toml
[submission.rate_limit]
max_inflight = 4
backoff_base = "1s"
backoff_max = "30s"
circuit_window = 20
circuit_open_threshold = 0.5
```
## References
- Sony's gobreaker or similar for circuit breaker pattern
Contributor guide
Assessment
This issue has not been assessed yet.