hardbyte / hardbyte/awa

Adaptive claimer autoscaling (experiment-gated, opt-in)

Open
#380 0 comments 0 reactions 0 assignees View on GitHub
feature research
Dominant language
Rust
Stars
32
Forks
5
Avg merge
15h 21m
Merged PRs (30d)
21

Description

## Motivation

The single-shard/default-shape ceiling is real: the 2026-06-07 worst-case run accepted ~9,982 jobs/s enqueue but completed only ~4,016/s (depth peaked 1.26M) with one claimer. The remedy exists (`claimers`, `enqueue_shards`, `PartitionedQueue`) but requires hand-tuning. A bounded controller that adjusts per-queue claimers within `[1, max]` on a lane-lag signal could make the default shape absorb bursts without operators reading the throughput-lever docs first.

## Scope (experiment first — roadmap E4)

- Prototype: lag-signal controller vs static settings across steady / bursty / skewed load shapes.
- **Decision rule**: ship opt-in only if the controller shows no oscillation, no ordering-contract violation, and ≥1.5× default-shape completion on the burst shape. Otherwise close this issue with the data and ship documented tuning presets instead.
- If shipped: opt-in per queue (`claimers = auto(max)`), metrics for controller decisions, docs positioning it relative to `enqueue_shards` (which remains the semantic switch — this knob must never change ordering semantics).

## Acceptance

- E4 results recorded here with run ids (benchmark-repo convention).
- Either: opt-in feature merged with controller-behaviour tests, or issue closed with evidence + presets documented in `configuration.md`.

## References

Roadmap: `docs/0.7-roadmap.md` (WS-2, NEW-7; experiment E4). Evidence: #169 comment 2026-06-07 (`custom-20260607T104130Z-9ad735`). Related: ADR-025, ADR-031, #341.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with docs/0.7-roadmap.md and ADR-025/ADR-031, then review the benchmark-repo results and referenced run #169 comment (`custom-20260607T104130Z-9ad735`). Done means recording E4 run IDs and either merging the opt-in feature with controller-behaviour tests or documenting evidence and tuning presets in configuration.md.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, rust
Domain
backend, distributed-systems, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.