✨ Add acquire mode for administrative block/bypass control
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 6h 51m
- Merged PRs (30d)
- 104
Description
Status
BLOCKED shipped early as ADR-125
disabledin v0.12.0 (#438). This issue now covers only what remains: BYPASSED (always admit, do not subtract tokens, still count consumption) and, if still wanted, a system-level block.What ADR-125 delivered, and how it differs from the BLOCKED sketch below:
This issue's BLOCKED sketch What shipped (ADR-125, v0.12.0) acquire_modeattribute on config itemsTri-state disabledattribute (true/false/ absent = inherit) on resource and entity config itemsResolved from cached config on the hot path Uncached Repository.resolve_disabled()on the slow path only; fast path enforced by adisabledstamp on bucket items andattribute_not_exists(disabled)in the speculativeUpdateItemcondition. Disabling is eager: config write + fan-out stamp over every existing bucketSystem > Resource > Entity hierarchy Entity(resource) > Entity( _default_) > Resource only. No system-level block (ADR-125 scopes it out)RateLimitExceededwithreason="blocked",retry_after_seconds=NoneDedicated ResourceDisabled(directZAELimiterErrorsubclass, not aRateLimitError, no retry hint — 403 semantics, not 429). Always propagates regardless ofon_unavailableset_*_acquire_mode/get_*/delete_*methodsdisable_resource()/enable_resource()/clear_resource_disabled(),disable_entity()/enable_entity()/clear_entity_disabled()onRepository;disabled=keyword onset_resource_defaults()/set_limits()system|resource|entity set-acquire-modeCLIzae-limiter resource|entity disable|enable|clear-disabledCLI;disabled:key in the declarative YAML manifest (#405) andDisabledproperty onCustom::ZaeLimiterLimitsAny BYPASSED design must be an extension of ADR-125's tri-state, not a parallel
acquire_modeattribute. See Design Constraints below.
Problem or Use Case
Operators need the ability to administratively control rate limit enforcement at the entity, resource, or system level, independent of token bucket state. Current use cases include:
Block a misbehaving entity: An API key is abusing the system; the operator wants to immediately deny all requests without waiting for buckets to drain.Shipped —disable_entity()(ADR-125).- Bypass limits for privileged entities: An admin user or internal service needs unlimited access to a resource while still tracking usage.
Emergency kill switch: Temporarily block all access to a specific resource during an incident.Shipped at resource level —disable_resource()(ADR-125). A system-level kill switch (block every resource in a namespace at once) remains open; see below.- Grace period for new entities: Allow a new entity to operate without rate limiting during onboarding.
Today, on_unavailable controls behavior when DynamoDB is unreachable, and ADR-125 disabled provides administrative deny, but there is no mechanism to administratively waive rate limit enforcement while still tracking usage. The only workaround is setting extremely high limits, which still consumes DynamoDB capacity for bucket operations and doesn't provide a clean audit trail.
Proposed Solution
Introduce an AcquireMode enum that controls how acquire() enforces limits:
# ORIGINAL SKETCH — superseded for BLOCKED by ADR-125; retained for history.
class AcquireMode(str, Enum):
ENFORCE = "enforce" # Normal: check token buckets (default)
BLOCKED = "blocked" # Administrative block: always deny -> shipped as `disabled: true`
BYPASSED = "bypassed" # Administrative bypass: always allow, track usage -> REMAINING
Remaining scope: extend ADR-125's tri-state disabled into a three-valued mode (working name) so that the same config attribute, the same resolution walk, and the same bucket-stamp enforcement mechanism cover both deny and waive:
| Stored value | Meaning |
|---|---|
| absent | inherit from the next level up (unchanged from ADR-125) |
disabled / true |
administrative block (unchanged from ADR-125) |
false |
explicit enable / carve-out (unchanged from ADR-125) |
bypass |
new — always admit, never subtract tokens, still count consumption |
Whether this is a widening of the existing disabled attribute's value set or a sibling attribute that resolve_disabled() learns to read is an implementation choice for the ADR that supersedes/extends ADR-125 — but it must be one resolution walk, not two.
Mode Behavior
| Mode | Token bucket (b_*_tk) |
Consumption counter (b_*_tc) |
Capacity check | DynamoDB write | Status |
|---|---|---|---|---|---|
ENFORCE (inherit / false) |
Subtract consumed | Increment | Yes — can reject | Yes | Current behaviour |
BLOCKEDdisabled: true |
Untouched | Untouched | N/A — reject immediately | No (fast path: conditional UpdateItem fails on attribute_not_exists(disabled), 0 WCU) |
Shipped — ADR-125, #438, v0.12.0 |
BYPASSED |
Untouched | Increment | No — always succeed | Yes (counter only) | Open |
Key design decision (unchanged): BYPASSED does NOT subtract tokens from the bucket, only increments the consumption counter. This means:
- Usage snapshots still reflect actual consumption for bypassed entities
- Switching from
BYPASSED→ENFORCErequires no bucket reset — the bucket is at its natural refill state - All mode transitions are clean with no reset logic needed
Design Constraints (post-ADR-125)
The BYPASSED design must answer these before implementation:
- One tri-state, not two attributes.
bypassis a fourth value of the ADR-125 resolution walk (entity(resource) → entity(_default_) → resource; first explicit value wins). No parallelacquire_modeattribute.resolve_disabled()(or its successor) returns the effective mode; thedisabledstamp on bucket items becomes a mode stamp. - Fast-path condition. ADR-125 enforces deny on the speculative fast path via
attribute_not_exists(disabled)in the conditionalUpdateItem, with no config read. BYPASSED must state what the fast path does when the bucket carries abypassstamp: thetk >= :consumedguard must be skipped (always admit) while theADD b_*_tc :consumedcounter increment still executes andADD b_*_tk -:consumedis omitted. This is a differentUpdateExpressionshape, so the client must know the mode before building the write — which means either (a) the stamp is cached inRepository._entity_cachealongsidecascade/parent_id/shard_countsand populated from theALL_NEW/ALL_OLDreturn values, or (b) the bypass write is unconditional and the stamp is consulted server-side via aConditionExpressionthat admits whenmode = :bypass. Option (b) still subtracts tokens unless the expression is chosen per mode, so (a) is the likely shape. The design must quantify the extra RT/RCU/WCU on the first acquire per entity (cache miss) and confirm zero extra cost on the warm path. - Eager fan-out on transition. Setting/clearing
bypassre-stamps every existing bucket for the scope, exactly asdisable_*()does today (O(buckets) writes, two-pass GSI3 discovery, same known race window).set_resource_defaults(disabled=...)/set_limits(disabled=...)accept the new value.delete_limits()/delete_resource_defaults()re-run fan-out against the newly resolved mode. - Interaction with #455's declared-scope rule. #455 makes
consumethe declared scope of a lease: aLeaseEntryexists only for limits named inconsume, on both paths. Underbypass, the counter increment (b_*_tc) must be applied to exactly the limits inconsume, andlease.adjust()/consume()/release()on a bypassed lease must still update the counter (and nevertk) for keys in the declared scope, and raise on undeclared keys the same way #455 specifies. Bypass does not widen or narrow the declared scope. - Cascade. Each entity resolves its mode independently (unchanged). Consistent with ADR-125, an entity-level
bypasson a cascading child does not extend to its parent: the parent's bucket is written with the parent's own mode. Child=bypass, Parent=enforce → child always passes, parent checked normally; if the parent rejects, no writes for either (write-on-enter invariant). Child=bypass, Parent=disabled →ResourceDisabledwithentity_id= parent. - Provisioner parity.
src/zae_limiter_provisioner/fanout.pymirrors the async fan-out; the YAMLdisabled:key (and the CFNDisabledproperty) must accept the new value, and the per-entity-override preservation rule from ADR-125 (an apply that re-asserts an unchanged resource-level value never clobbers an out-of-band entity carve-out) must hold forbypasstoo. - System-level block (optional, separate decision). ADR-125 scoped out
systemdeliberately. If a namespace-wide kill switch is still wanted, it needs its own justification:set_system_defaults()has no fan-out today, and a system-level stamp touches every bucket in the namespace (GSI4 discovery). Decide whether this is in scope for #311 or a separate issue before implementation.
Configuration Hierarchy
Acquire mode is stored as an acquire_mode attribute on existing config records (no new items). Resolution follows the same four-tier hierarchy as limits: Entity (resource-specific) > Entity (default) > Resource > System > ENFORCE (default)
Superseded by ADR-125: resolution is Entity (resource-specific) > Entity (_default_) > Resource, first explicit value wins, no system level. bypass joins that walk as a new explicit value.
Hot Path
Original cached-config sketch (Phase 1 READ / Phase 2 CHECK / Phase 3 WRITE). Superseded: ADR-125 enforces on the speculative fast path via the bucket stamp with no config read. BYPASSED must follow the same pattern — see Design Constraint 2.
Cascade Behavior
See Design Constraint 5. Child=BLOCKED, Parent=ENFORCE → child rejected immediately, parent never reached. Shipped — ResourceDisabled raised from the child stamp; parent write never issued. Child=ENFORCE, Parent=BLOCKED → child passes, parent rejected → raise, no writes for either. Shipped — ResourceDisabled with entity_id = parent; child write compensated per write-on-enter.
API Surface
Python API (on Repository, matching ADR-125 placement — not RateLimiter):
| Level | Set | Clear | Status |
|---|---|---|---|
set_system_acquire_mode(mode) / get_system_acquire_mode() / delete_system_acquire_mode() |
Struck — no system level (ADR-125). Re-add only if Design Constraint 7 is accepted | ||
set_resource_acquire_mode(resource, mode) / get_resource_acquire_mode / delete_resource_acquire_mode |
Shipped as disable_resource() / enable_resource() / clear_resource_disabled() |
||
set_acquire_mode(entity_id, mode, resource=None) / get_acquire_mode / delete_acquire_mode |
Shipped as disable_entity() / enable_entity() / clear_entity_disabled() |
||
| Resource (bypass) | bypass_resource(resource) (working name) |
clear_resource_disabled(resource) (existing — clears any explicit value) |
Open |
| Entity (bypass) | bypass_entity(entity_id, resource=None) (working name) |
clear_entity_disabled(entity_id, resource=None) (existing) |
Open |
| Any | set_resource_defaults(..., disabled="bypass") / set_limits(..., disabled="bypass") — or whatever the extended keyword is named |
Open | |
list_resources_with_acquire_mode() / list_entities_by_acquire_mode(mode, resource=None) |
Deferred — ADR-125 shipped no list query and no sparse GSI; get_resource_defaults() / get_limits() expose the explicit value. Reconsider only if an operator use case surfaces |
SyncRepository exposes matching generated methods.
CLI:
# ~~System level~~ — struck (no system level in ADR-125; see Design Constraint 7)
# Resource level
# zae-limiter resource disable|enable|clear-disabled RESOURCE_NAME <- shipped (ADR-125)
zae-limiter resource bypass RESOURCE_NAME # open (working name)
# Entity level
# zae-limiter entity disable|enable|clear-disabled ENTITY_ID [--resource R] <- shipped (ADR-125)
zae-limiter entity bypass ENTITY_ID [--resource R] # open (working name)
# `resource get-defaults` / `entity get-limits` already show `Status: DISABLED` /
# `Status: enabled (explicit override)`; add `Status: BYPASSED`.
Declarative manifest (#405): the existing disabled: key on resources.<name> and entities.<id>.resources.<name> accepts the new value; the CFN Disabled property round-trips it.
DynamoDB Storage
Mode stored on existing config records — no new items — already true for ADR-125's disabled:
| Level | PK | SK | Attribute | Status |
|---|---|---|---|---|
{ns}/SYSTEM# |
#CONFIG |
acquire_mode |
Struck — no system level | |
| Resource | {ns}/RESOURCE#{resource} |
#CONFIG |
disabled (tri-state) |
Shipped — ADR-125 |
| Entity-wide | {ns}/ENTITY#{id} |
#CONFIG#_default_ |
disabled (tri-state) |
Shipped — ADR-125 |
| Entity+resource | {ns}/ENTITY#{id} |
#CONFIG#{resource} |
disabled (tri-state) |
Shipped — ADR-125 |
| Bucket stamp | {ns}/BUCKET#{id}#{resource}#{shard} |
#STATE |
disabled (present = blocked) |
Shipped — ADR-125 |
| Bucket stamp | same | same | bypass value on the same stamp (or a sibling — the ADR decides) |
Open |
Sparse GSI4 for list queries (only indexes records where — struck. GSI4 is now taken by namespace-scoped item discovery (acquire_mode is set)purge_namespace()), ADR-125 shipped no list index, and no operator use case has asked for one.
Exception Enrichment
Superseded — ADR-125 raises a dedicated RateLimitExceeded gains a reason field to distinguish administrative blocks from capacity exhaustion.ResourceDisabled (not a RateLimitError, no retry_after_seconds) so callers map it to 403 rather than 429. Bypass never raises, so no exception change remains in scope.
Entity Model
Deferred — ADR-125 did not add a field to Entity gains an acquire_mode field.Entity; the explicit value is read from config (get_limits() / get_resource_defaults()). Reconsider only if the bypass ADR needs it.
Acceptance Criteria
Shipped as ADR-125 (#438, v0.12.0) — retained for traceability
- Tri-state
disabledstored on resource and entity config items (entity default + entity+resource) - Resolution walk: Entity+resource > Entity
_default_> Resource, first explicit value wins - Block enforced on the speculative fast path via bucket stamp +
attribute_not_exists(disabled); raisesResourceDisabledbefore any token write - Cascade: child stamp checked first; parent stamp raises
ResourceDisabledwithentity_id= parent -
disable_*/enable_*/clear_*_disabledonRepositoryandSyncRepository;disabled=keyword onset_resource_defaults()/set_limits() -
resource|entity disable|enable|clear-disabledCLI;Status:line inget-defaults/get-limits -
disabled:in YAML manifest andDisabledonCustom::ZaeLimiterLimits; provisioner fan-out preserves entity carve-outs - Audit events for disable/enable/clear at resource and entity level
- CLAUDE.md, ADR-125, and user docs updated
Remaining — BYPASSED
Design (ADR that extends ADR-125)
- ADR filed (new number; does not edit ADR-125) that adds
bypassas a value of the existing tri-state and answers Design Constraints 1–6 above, with RT/RCU/WCU quantified for cold-cache and warm-cache acquire - Decision recorded on Design Constraint 7 (system-level block): in scope here, split to a new issue, or rejected
Core
-
bypassvalue stored on the same resource / entity config attribute ADR-125 uses (no parallelacquire_mode) -
resolve_disabled()(or its renamed successor) returns the effective mode from the single walk;bypassis an explicit value that stops the walk - Eager fan-out stamps every existing bucket for the scope on set/clear, same two-pass discovery as
disable_* - Speculative fast path under
bypass: admits without thetk >= :consumedguard, omitsADD b_*_tk -:consumed, still issuesADD b_*_tc :consumed— 0 RCU, 1 WCU per bucket, no config read - Slow path under
bypass: same write shape;_commit_initial()never subtractstk -
lease.adjust()/consume()/release()underbypassupdateb_*_tconly, for keys in the declared scope (#455); undeclared keys behave exactly as #455 specifies -
_rollback()underbypasscompensates the counter only - Cascade: each entity resolves independently; child
bypassdoes not extend to the parent; parentdisabledstill raisesResourceDisabled(entity_id=parent) - Bucket TTL rules (#271/#296) unchanged by mode
-
ENFORCE(inherit /false) behaviour unchanged;disabled: truebehaviour unchanged
API
-
bypass_resource(resource)/bypass_entity(entity_id, resource=None)(or the names the ADR settles on) onRepository, returning the bucket-write count likedisable_* - Existing
clear_resource_disabled()/clear_entity_disabled()clearbypasstoo -
set_resource_defaults()/set_limits()accept the new value via the existing keyword; explicit pass fans out immediately -
delete_limits()/delete_resource_defaults()re-run fan-out against the newly resolved mode (extends the ADR-125 rule) -
SyncRepositorygenerated methods match
CLI
-
resource bypass RESOURCE_NAMEandentity bypass ENTITY_ID [--resource R] -
resource get-defaults/entity get-limitsprintStatus: BYPASSED
Declarative limits (#405)
- YAML
disabled:key (onresources.<name>andentities.<id>.resources.<name>) accepts the new value;Custom::ZaeLimiterLimitsDisabledround-trips it -
src/zae_limiter_provisioner/fanout.pymirrors the new stamp and preserves entity carve-outs on re-apply
Observability
- Audit events recorded for bypass set/clear at resource and entity level (
$RESOURCE:{name}/ entity id, per ADR-106) - Usage snapshots for bypassed entities reflect real consumption (counter-driven, per issue #179 rule — no
tkdelta derivation)
Tests
- Unit: fast path and slow path under
bypass—tkunchanged,tcincremented, 0 RCU / 1 WCU on warm path - Unit: resolution walk with
bypassat each level, including entitybypasscarve-out under adisabledresource and the reverse - Unit: cascade matrix (child bypass / parent enforce, child bypass / parent disabled, child enforce / parent bypass)
- Unit:
adjust()/consume()/release()/ rollback underbypasstouchtconly and respect #455 declared scope - Unit: fan-out on set/clear and on
delete_*re-resolution - Integration (LocalStack): stamp visible on every shard; aggregator refill (#317) does not fight the stamp
- Generated sync tests match
Documentation
- CLAUDE.md "Disabling Resources and Entities (ADR-125)" section extended to cover
bypass; DynamoDB writer table gains the bypass rows -
docs/guide/anddocs/cli.mdupdated;docs-updaterrun
Remaining — System-level block (only if Design Constraint 7 is accepted)
-
disable_system()/enable_system()/clear_system_disabled()with namespace-wide fan-out via GSI4 -
system disable|enable|clear-disabledCLI - Manifest
system.disabled:accepted (ADR-125 explicitly rejects this today — needs the ADR to reverse it)
Alternatives Considered
Mode as parameter on set_limits(): Simpler API surface (fewer methods) but conflates limit configuration with enforcement control. ADR-125 chose both: dedicated disable_* methods and a disabled= keyword on the setters (with a "preserve" sentinel default). Bypass follows the same pattern.
Separate block() / bypass() / unblock() API: Breaks the ADR-125 went this way (set/get/delete naming convention.disable_resource() / enable_resource() / clear_resource_disabled()); the bypass methods should match it, not the original set_*_acquire_mode sketch.
Mode on META/bucket records (read every acquire): Rejected in favour of cached config. Reversed by ADR-125: the bucket stamp is the enforcement mechanism, precisely because the speculative fast path never reads config. Cached-config resolution would have left a warm bucket admitting traffic against a disabled resource. Bypass inherits this decision.
Subtract tokens during bypass + reset on mode change: Would require bulk bucket resets when switching resource-level bypass back to enforce. The "don't subtract, only track counter" approach eliminates all reset logic and makes mode transitions clean at every level. Still the decision.
Boolean blocked flag: Too limited — doesn't support bypass. ADR-125 shipped a tri-state boolean, which supports inherit / explicit carve-outs. It is limited only in that it cannot express bypass — hence this issue's remaining scope is to widen its value set rather than add a parallel attribute.
Parallel acquire_mode attribute alongside disabled: Rejected. Two attributes with two resolution walks and two stamps would let them disagree (e.g. disabled: true at resource, acquire_mode: bypassed at entity) with no defined precedence, and would double the fan-out cost on every transition.
Related
- ADR-125 — Resource and Entity Disable (
docs/adr/125-resource-disable.md), Accepted 2026-08-30 - #438 — PR that shipped resource/entity disable (v0.12.0)
- #455 —
consumeas the declared scope of a lease;adjust()on undeclared keys - #405 — Declarative limits (YAML manifest + provisioner) that
disabled:/ bypass round-trip through - #309 — Write-on-enter invariant (no writes on rejection; bypass writes counter only)
- #179 — Counter-driven consumption (bypass must not rely on
tkdeltas)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with docs/adr/125-resource-disable.md and the existing Repository disabled path described in the issue, then compare the async and provisioner fan-out in src/zae_limiter_provisioner/fanout.py. Define the ADR extension for bypass before implementation, including cache and bucket-stamp behavior, cascade and lease scope, and configuration/API names. Done means bypass admits requests, increments only declared consumption counters, preserves existing disabled behavior, and has matching provisioner and manifest support.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100