citusdata / citusdata/citus_docs
Improvements to Scaling out Data Ingestion section
- Dominant language
- CSS
- Stars
- 58
- Forks
- 60
- PR merge metrics
- No merged PRs in 30d
Description
We recently merged in a pull request (#402) to make improvements to the "Scaling out Data Ingestion" section in our docs.
With #402, we set more concrete expectations on what users can expect from their Citus clusters in terms of write throughput. We also provide repeatable benchmark steps through pgbench.
We could still improve on this section however. For #402, we could make the following two changes:
* We can add an architectural diagram that illustrates the benchmarking set-up: pgbench -> Citus coordinator -> Citus workers -> EBS (network attached storage). This diagram would better explain why network latency becomes a bottleneck in write benchmarks
* Provide pgbench installation steps for more recent pgbench packages and update the insert.sql / update.sql instructions accordingly
The following ideas aren't related to #402, but they would improve the Scaling out Data Ingest section:
* Rename the Scaling out Data Ingest section to Write Benchmarks
* We can provide benchmarks that are more reflective of production workloads and explain in detail the different trade-offs one would observe.
* Once we merge in the multi-row insert changes (#455), we could revise this section to progress as "single row changes", "multi-row changes", and "batched changes" (or similar).
* We can make edits to "COPY" and "Citus MX (Beta)" sections. In particular, we could explain the tradeoffs on the application integration side about using Citus MX.
Contributor guide
Assessment
This issue has not been assessed yet.