RFC: Evaluate replacing Redis with DragonflyDB as the caching backend
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 1.6k
- Forks
- 438
- PR merge metrics
- No merged PRs in 30d
Description
Hi @e2b-dev/infra maintainers,
In the current infra, Redis (via github.com/redis/go-redis/v9) backs several low-latency caches — auth cache, template metadata & alias cache, and the RedisSandboxCatalog in client-proxy (see packages/client-proxy, packages/api/internal/cache/templates, PRs #1914/#1915/#2289). Given the characteristics of DragonflyDB (https://github.com/dragonflydb/dragonfly ), I'd like to open an RFC on whether there's interest in evaluating it as a drop-in caching backend.
Why it looks relevant:
Wire/Client compatibility — DragonflyDB is 100% Redis protocol & command compatible; the existing go-redis/v9 clients would work by only swapping the endpoint, no app-code changes.
Throughput & footprint — official benchmarks show ~25X the QPS of single-process Redis on the same instance (3.8M QPS on c6gn.16xlarge) and up to 80% lower resource usage for the same workload, with no memory spike during snapshotting. This could help with the high churn rate caused by frequent sandbox create/destroy cycles.
Migration path — supports live REPLICAOF replication and RDB import, so a cutover can be done without code changes.
I'd like to understand the project's stance:
Is DragonflyDB already on the radar for cache backing, or ruled out deliberately (e.g. reliance on Redis Cluster slot semantics, specific Lua/PubSub behavior, or Valkey being the chosen direction)?
Do any e2b Redis usages touch commands/features Dragonfly doesn't implement that would block a switch?
If evaluated, would shadow-traffic on a non-critical path (e.g. template alias cache) be the preferred first step?
Not proposing immediate action — just collecting maintainer & community input on design direction. Thanks!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing Redis usage in packages/client-proxy and packages/api/internal/cache/templates, along with PRs #1914, #1915, and #2289. Inventory the commands and behaviors those caches rely on, then compare them with DragonflyDB compatibility and a possible shadow-traffic path. Done means documenting the compatibility findings and a clear evaluation recommendation for maintainers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, redis
- Domain
- backend, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100