hardbyte / hardbyte/postgresql-job-queue-benchmarking
Standardize queue-depth and offered-load measurement across adapters
- Ngôn ngữ chính
- HTML
- Star
- 2
- Fork
- 0
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Mô tả
## Problem
Adapters today report queue depth from inconsistent vantage points,
which makes the cross-system depth column in `COMPARISON.md` apples to
oranges:
- Some adapters sample **available-to-claim** depth (e.g. awa reads
`queue_lanes.available_count` — jobs ready for any worker to pick
up).
- Some adapters sample **consumer-pending** depth (e.g. pgque reads
`pgque.get_consumer_info(queue, consumer).pending_events` — events
the named consumer hasn't acknowledged yet, which reads 0 whenever
the consumer keeps up regardless of stream-side backlog).
- Others may use **total ingestable** counts that include scheduled /
retrying / waiting-external rows, mixing dispatch readiness with
backlog.
This becomes a real interpretation problem under pressure scenarios
(idle-in-tx, active-readers, high-load) where backlog growth is the
*signal* the bench is trying to capture. A measurement that reads 0
when the consumer is keeping up hides the back-pressure dynamic, and
any cross-system depth chart effectively asks viewers to know which
shape each adapter sampled.
## Compounding problem: offered load vs achieved load
Some adapters' producer paths cap below `--producer-rate` because the
per-insert cost exceeds the budget at the configured concurrency. When
that happens the queue may genuinely never accumulate — but only
because the producer was the bottleneck, not because the engine kept
up. The bench currently treats `--producer-rate` as the offered load
even when `enqueue_rate` is materially below it.
The two issues compound: a system whose producer ceiling sits at
~110/s when offered 200/s will report depth ≈ 0 forever, and a
viewer can't tell whether that's "engine kept up" or "producer
bottleneck masked the test."
## Proposal
1. **Standardize the depth metric.** Pick one definition and document
it in `CONTRIBUTING_ADAPTERS.md`. Strawman: *jobs that have been
accepted by the engine but have not yet started executing* (i.e.
ready-to-claim, exclusive of scheduled / waiting / retrying). Each
adapter implements this; existing per-state metrics stay alongside
for diagnostics.
2. **Add an offered-vs-achieved load check.** The orchestrator already
knows the producer's target rate; surface a per-phase
`producer_attainment` metric (achieved enqueue_rate / target rate)
so the comparison can flag systems whose producer was the
bottleneck.
3. **Update `COMPARISON.md`** to call out attainment alongside
throughput when it's <90% of target, so a "kept up" reading isn't
silently misleading.
## Out of scope
Per-state diagnostic metrics (`scheduled_depth`, `retryable_depth`,
`running_depth`) stay as they are — useful for adapter-level debug,
not the cross-system headline.
## Provenance
Surfaced during the 2026-04-30 5-min long-tx comparison
(`results/custom-20260430T183931Z-ead316`): pgque reported
`queue_depth = 0` across every phase including idle-in-tx + readers,
which read at face value as "queue never backed up" but was actually
"producer never managed to offer the configured load because per-insert
cost capped throughput at ~110-180/s." awa's depth was 5-20 jobs across
the same phases at the same target offered load. Both readings are
internally consistent with their adapter's sampling; neither is wrong;
they're just different shapes. The bench should make that explicit.
Hướng dẫn đóng góp
Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này
Hướng nghiên cứu
Start by reading CONTRIBUTING_ADAPTERS.md, COMPARISON.md, and the orchestrator's producer-rate and per-phase metric paths; inspect adapter implementations that report queue depth. Define and document one ready-to-claim depth measure, add producer_attainment, and update the comparison output when attainment is below 90%. Done means adapters use the agreed metric and comparisons expose when producer bottlenecks mask backlog.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- postgresql
- Lĩnh vực
- databases, documentation, performance
- Loại issue
- Tính năng
- Độ khó
- 5/5
- Thời gian dự kiến
- Hơn một tuần
- Mức độ hoạt động
- Ít trao đổi
- Độ rõ ràng
- Khá rõ ràng
- Mức phù hợp với người mới
- 35/100