hardbyte / hardbyte/postgresql-job-queue-benchmarking

Standardize queue-depth and offered-load measurement across adapters

Open
#2 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
2
Forks
0
PR merge metrics
No merged PRs in 30d

Description

## Problem

Adapters today report queue depth from inconsistent vantage points,
which makes the cross-system depth column in `COMPARISON.md` apples to
oranges:

- Some adapters sample **available-to-claim** depth (e.g. awa reads
`queue_lanes.available_count` — jobs ready for any worker to pick
up).
- Some adapters sample **consumer-pending** depth (e.g. pgque reads
`pgque.get_consumer_info(queue, consumer).pending_events` — events
the named consumer hasn't acknowledged yet, which reads 0 whenever
the consumer keeps up regardless of stream-side backlog).
- Others may use **total ingestable** counts that include scheduled /
retrying / waiting-external rows, mixing dispatch readiness with
backlog.

This becomes a real interpretation problem under pressure scenarios
(idle-in-tx, active-readers, high-load) where backlog growth is the
*signal* the bench is trying to capture. A measurement that reads 0
when the consumer is keeping up hides the back-pressure dynamic, and
any cross-system depth chart effectively asks viewers to know which
shape each adapter sampled.

## Compounding problem: offered load vs achieved load

Some adapters' producer paths cap below `--producer-rate` because the
per-insert cost exceeds the budget at the configured concurrency. When
that happens the queue may genuinely never accumulate — but only
because the producer was the bottleneck, not because the engine kept
up. The bench currently treats `--producer-rate` as the offered load
even when `enqueue_rate` is materially below it.

The two issues compound: a system whose producer ceiling sits at
~110/s when offered 200/s will report depth ≈ 0 forever, and a
viewer can't tell whether that's "engine kept up" or "producer
bottleneck masked the test."

## Proposal

1. **Standardize the depth metric.** Pick one definition and document
it in `CONTRIBUTING_ADAPTERS.md`. Strawman: *jobs that have been
accepted by the engine but have not yet started executing* (i.e.
ready-to-claim, exclusive of scheduled / waiting / retrying). Each
adapter implements this; existing per-state metrics stay alongside
for diagnostics.
2. **Add an offered-vs-achieved load check.** The orchestrator already
knows the producer's target rate; surface a per-phase
`producer_attainment` metric (achieved enqueue_rate / target rate)
so the comparison can flag systems whose producer was the
bottleneck.
3. **Update `COMPARISON.md`** to call out attainment alongside
throughput when it's <90% of target, so a "kept up" reading isn't
silently misleading.

## Out of scope

Per-state diagnostic metrics (`scheduled_depth`, `retryable_depth`,
`running_depth`) stay as they are — useful for adapter-level debug,
not the cross-system headline.

## Provenance

Surfaced during the 2026-04-30 5-min long-tx comparison
(`results/custom-20260430T183931Z-ead316`): pgque reported
`queue_depth = 0` across every phase including idle-in-tx + readers,
which read at face value as "queue never backed up" but was actually
"producer never managed to offer the configured load because per-insert
cost capped throughput at ~110-180/s." awa's depth was 5-20 jobs across
the same phases at the same target offered load. Both readings are
internally consistent with their adapter's sampling; neither is wrong;
they're just different shapes. The bench should make that explicit.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.