hardbyte / hardbyte/postgresql-job-queue-benchmarking

Standardize queue-depth and offered-load measurement across adapters

Ouverte
#2 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
HTML
Étoiles
2
Forks
0
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

## Problem

Adapters today report queue depth from inconsistent vantage points,
which makes the cross-system depth column in `COMPARISON.md` apples to
oranges:

- Some adapters sample **available-to-claim** depth (e.g. awa reads
`queue_lanes.available_count` — jobs ready for any worker to pick
up).
- Some adapters sample **consumer-pending** depth (e.g. pgque reads
`pgque.get_consumer_info(queue, consumer).pending_events` — events
the named consumer hasn't acknowledged yet, which reads 0 whenever
the consumer keeps up regardless of stream-side backlog).
- Others may use **total ingestable** counts that include scheduled /
retrying / waiting-external rows, mixing dispatch readiness with
backlog.

This becomes a real interpretation problem under pressure scenarios
(idle-in-tx, active-readers, high-load) where backlog growth is the
*signal* the bench is trying to capture. A measurement that reads 0
when the consumer is keeping up hides the back-pressure dynamic, and
any cross-system depth chart effectively asks viewers to know which
shape each adapter sampled.

## Compounding problem: offered load vs achieved load

Some adapters' producer paths cap below `--producer-rate` because the
per-insert cost exceeds the budget at the configured concurrency. When
that happens the queue may genuinely never accumulate — but only
because the producer was the bottleneck, not because the engine kept
up. The bench currently treats `--producer-rate` as the offered load
even when `enqueue_rate` is materially below it.

The two issues compound: a system whose producer ceiling sits at
~110/s when offered 200/s will report depth ≈ 0 forever, and a
viewer can't tell whether that's "engine kept up" or "producer
bottleneck masked the test."

## Proposal

1. **Standardize the depth metric.** Pick one definition and document
it in `CONTRIBUTING_ADAPTERS.md`. Strawman: *jobs that have been
accepted by the engine but have not yet started executing* (i.e.
ready-to-claim, exclusive of scheduled / waiting / retrying). Each
adapter implements this; existing per-state metrics stay alongside
for diagnostics.
2. **Add an offered-vs-achieved load check.** The orchestrator already
knows the producer's target rate; surface a per-phase
`producer_attainment` metric (achieved enqueue_rate / target rate)
so the comparison can flag systems whose producer was the
bottleneck.
3. **Update `COMPARISON.md`** to call out attainment alongside
throughput when it's <90% of target, so a "kept up" reading isn't
silently misleading.

## Out of scope

Per-state diagnostic metrics (`scheduled_depth`, `retryable_depth`,
`running_depth`) stay as they are — useful for adapter-level debug,
not the cross-system headline.

## Provenance

Surfaced during the 2026-04-30 5-min long-tx comparison
(`results/custom-20260430T183931Z-ead316`): pgque reported
`queue_depth = 0` across every phase including idle-in-tx + readers,
which read at face value as "queue never backed up" but was actually
"producer never managed to offer the configured load because per-insert
cost capped throughput at ~110-180/s." awa's depth was 5-20 jobs across
the same phases at the same target offered load. Both readings are
internally consistent with their adapter's sampling; neither is wrong;
they're just different shapes. The bench should make that explicit.

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.