hardbyte / hardbyte/postgresql-job-queue-benchmarking

Apply producer in-flight tracking to poller-based adapters before next cross-system sweep

オープン
#31 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
HTML
スター
2
フォーク
0
PR マージ指標
30日以内にマージされた PR はありません

説明

## Context

PR #30 fixed an overshoot in awa-bench's depth-target producer: the
producer now tracks its own contribution since the last depth-poll
snapshot, so bursts during the ~200 ms stale window self-limit.

## Per-adapter shape

| Adapter | Depth source | Status |
|---|---|---|
| pgboss | live \`getQueueStats()\` inside producer loop | accurate (already tracks target) |
| pgmq | live \`queue_depth()\` inside producer loop | accurate (already tracks target) |
| **awa** | atomic + in-flight tracker (this fix) | accurate |
| oban | ETS table populated by 200 ms-cadence poller | **overshoots** |
| river | atomic populated by 200 ms-cadence poller | **overshoots** |
| pgque | variable populated by 200 ms-cadence poller | **overshoots** |
| absurd | variable populated by 200 ms-cadence poller | **overshoots** |
| procrastinate | variable populated by 200 ms-cadence poller | **overshoots** |

## Why it matters

At a given \`target_depth\` setting, the five overshoot-adapters end up
operating at higher effective backlog than the three accurate-adapters.
Higher backlog means more queued work, more scan cost on claim
queries, and (depending on the system) more lock contention or
table bloat. Cross-system throughput comparisons under depth-target
mode are not at the same operating point.

The pattern from PR #30 applies cleanly to all five poller-based
adapters:

1. Snapshot \`enqueued\` in the poller right after updating
\`queue_depth\`, keyed against a consistent moment.
2. In the producer, compute \`effective_depth = queue_depth +
(enqueued_now - enqueued_at_poll)\` and throttle against that.

## Acceptance criteria

- oban, river, pgque, absurd, procrastinate producers all track
target accurately under depth-target mode (median \`queue_depth\`
within ±20 % of target during steady-state soak).
- Each adapter's change is a self-contained PR with its own smoke
showing the before/after delta on the same hardware.
- Documented in CONTRIBUTING_ADAPTERS.md so future adapters get
the pattern right from the start.

## Out of scope

- The 200 ms cadence itself. PR #30's earlier commit established
that 200 ms is load-bearing for the depth-target controller;
loosening it regressed throughput by ~17 % and doubled depth.
The fix is the in-flight tracker, not a tighter poll.

Refs: PR #30, awa #289 (queue_counts_fast — the other half of the
depth-target hot path).

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

調査の方向性

まず PR #30 を確認して既存の in-flight tracking パターンを理解し、次に oban、river、pgque、absurd、procrastinate の poller と producer のパスを特定します。depth-target の動作を同一ハードウェアでの smoke 実行と比較し、定常状態のキュー深度が受け入れ範囲内にあることを検証して、アダプターパターンを CONTRIBUTING_ADAPTERS.md に記録します。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
postgresql
領域
backend, documentation, performance
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
静か
明瞭さ
おおむね明確
初心者へのやさしさ
45/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。