Improve txpool_content performance for high-frequency polling
- Dominant language
- Rust
- Stars
- 8
- Forks
- 9
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 6
Description
Summary
When polling txpool_content via the Ethereum JSON-RPC, I’m seeing a hard ceiling of ~200 transactions per second retrieved, even when the node’s actual mempool is much larger and incoming rate is higher.
This is likely due to the heavy JSON serialization and full-pool traversal in each RPC call. The endpoint works fine for occasional inspection, but is too slow for real-time monitoring or ingestion scenarios that require higher throughput.
Reproduction
1. Run reth with default RPC settings and a live mainnet sync.
2. Call txpool_content repeatedly (HTTP or WS eth_call) as fast as possible from a single client.
3. Measure transactions retrieved per second — result is consistently ~200 tx/s even with thousands of txs pending in the pool.
Expected behavior
• Ability to retrieve higher throughput from the mempool via RPC (e.g., closer to the actual incoming tx rate), especially for local/IPC clients.
• Option to fetch incremental changes instead of the full pool every time.
Why this matters
• Full-pool polling is currently the only way to get a complete snapshot of pending and queued txs via JSON-RPC.
• Many applications (analytics, MEV research, relays, custom ordering engines) need near-real-time mempool state but can’t efficiently use txpool_content due to this throughput cap.
Potential solutions
1. Add a streaming or incremental API — e.g., a txpool_diff or subscription that streams full tx bodies for new/removed txs (similar to eth_subscribe:newPendingTransactions but with bodies, not just hashes).
2. Expose a binary/IPC interface — avoid JSON encoding overhead for local clients.
3. Allow partial fetches — parameters to limit to N txs, specific senders, or time ranges to reduce traversal cost.
4. Optimize serialization — e.g., pre-encode pool entries to JSON or use more efficient traversal for RPC dumps.
Contributor guide
Research direction
Start by reproducing repeated txpool_content calls over HTTP and WS eth_call on a live mainnet sync, measuring transactions retrieved per second and pool size. Compare the full-pool traversal and JSON serialization costs described in the issue. Done means a concrete higher-throughput or incremental retrieval approach is implemented and its performance is measured against the reported ~200 tx/s ceiling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, blockchain, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100