hardbyte / hardbyte/postgresql-job-queue-benchmarking

Standardize queue-depth and offered-load measurement across adapters

オープン
#2 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
HTML
スター
2
フォーク
0
PR マージ指標
30日以内にマージされた PR はありません

説明

## Problem

Adapters today report queue depth from inconsistent vantage points,
which makes the cross-system depth column in `COMPARISON.md` apples to
oranges:

- Some adapters sample **available-to-claim** depth (e.g. awa reads
`queue_lanes.available_count` — jobs ready for any worker to pick
up).
- Some adapters sample **consumer-pending** depth (e.g. pgque reads
`pgque.get_consumer_info(queue, consumer).pending_events` — events
the named consumer hasn't acknowledged yet, which reads 0 whenever
the consumer keeps up regardless of stream-side backlog).
- Others may use **total ingestable** counts that include scheduled /
retrying / waiting-external rows, mixing dispatch readiness with
backlog.

This becomes a real interpretation problem under pressure scenarios
(idle-in-tx, active-readers, high-load) where backlog growth is the
*signal* the bench is trying to capture. A measurement that reads 0
when the consumer is keeping up hides the back-pressure dynamic, and
any cross-system depth chart effectively asks viewers to know which
shape each adapter sampled.

## Compounding problem: offered load vs achieved load

Some adapters' producer paths cap below `--producer-rate` because the
per-insert cost exceeds the budget at the configured concurrency. When
that happens the queue may genuinely never accumulate — but only
because the producer was the bottleneck, not because the engine kept
up. The bench currently treats `--producer-rate` as the offered load
even when `enqueue_rate` is materially below it.

The two issues compound: a system whose producer ceiling sits at
~110/s when offered 200/s will report depth ≈ 0 forever, and a
viewer can't tell whether that's "engine kept up" or "producer
bottleneck masked the test."

## Proposal

1. **Standardize the depth metric.** Pick one definition and document
it in `CONTRIBUTING_ADAPTERS.md`. Strawman: *jobs that have been
accepted by the engine but have not yet started executing* (i.e.
ready-to-claim, exclusive of scheduled / waiting / retrying). Each
adapter implements this; existing per-state metrics stay alongside
for diagnostics.
2. **Add an offered-vs-achieved load check.** The orchestrator already
knows the producer's target rate; surface a per-phase
`producer_attainment` metric (achieved enqueue_rate / target rate)
so the comparison can flag systems whose producer was the
bottleneck.
3. **Update `COMPARISON.md`** to call out attainment alongside
throughput when it's <90% of target, so a "kept up" reading isn't
silently misleading.

## Out of scope

Per-state diagnostic metrics (`scheduled_depth`, `retryable_depth`,
`running_depth`) stay as they are — useful for adapter-level debug,
not the cross-system headline.

## Provenance

Surfaced during the 2026-04-30 5-min long-tx comparison
(`results/custom-20260430T183931Z-ead316`): pgque reported
`queue_depth = 0` across every phase including idle-in-tx + readers,
which read at face value as "queue never backed up" but was actually
"producer never managed to offer the configured load because per-insert
cost capped throughput at ~110-180/s." awa's depth was 5-20 jobs across
the same phases at the same target offered load. Both readings are
internally consistent with their adapter's sampling; neither is wrong;
they're just different shapes. The bench should make that explicit.

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。