✨ Support utilization-based adaptive rate limits

Problem or Use Case

Users want rate limits that automatically scale based on resource utilization:

1. Off-peak scaling - When resource utilization is low (e.g., night time), users can automatically access more capacity without configuring time-based schedules
2. Fair distribution - Available capacity is distributed proportionally based on actual demand
3. Soft capacity targets - Resource capacity is advisory (for optimization), not a hard enforcement boundary

Currently, users must either:

• Set static limits that waste capacity during off-peak
• Implement complex time-based schedules that don't adapt to actual usage
• Build custom orchestration to monitor utilization and call set_limits()

Proposed Solution

Add utilization-based adaptive mode to limits:

# Define resource with advisory capacity (soft target, not enforced)
await limiter.set_resource_info(
    resource="gpt-4",
    estimated_capacity=100_000,  # TPM we expect to be available
)

# User has base limit that scales up when capacity is available
await limiter.set_limits(
    "user-123",
    resource="gpt-4",
    limits=[
        Limit.per_minute(
            "tpm",
            capacity=1000,           # Guaranteed base
            burst=10000,             # Max they can ever get
            adaptive="utilization",  # Scale based on resource utilization
        )
    ],
)

Behavior at acquire time:

Resource Utilization	Effective Limit	Notes
20% (night)	5000 TPM (5x base)	Capacity available, scale up
50%	2000 TPM (2x base)	Some headroom
100%	1000 TPM (1x base)	At capacity, use base
150% (over)	1000 TPM (1x base)	Over capacity, still use base (no reduction)

Key principle: Resource capacity is a scaling factor, not a ceiling. Users are never blocked because the resource is "over capacity" - they just get their base allocation.

Implementation Notes

Effective limit calculation:

utilization = current_resource_usage / estimated_capacity
scaling_factor = min(1.0 / utilization, burst / capacity) if utilization > 0 else burst / capacity
effective_capacity = capacity * scaling_factor  # Capped at burst

Resource utilization query:

• Use GSI2 (GSI2PK=RESOURCE#{resource}) to aggregate current usage
• Pull from recent usage snapshots (requires configurable snapshot intervals)

New storage:

• Resource metadata record: PK=RESOURCE#{name}, SK=#META
• Fields: estimated_capacity, adaptive_config

Success Criteria

• Define advisory capacity at resource level via set_resource_info()
• Limits with adaptive="utilization" scale based on current resource usage
• Effective limit never drops below base capacity
• Effective limit never exceeds burst
• CLI can display current resource utilization and effective limits
• Works with hierarchical limits (cascade mode)

Dependencies

• Configurable snapshot intervals - Need 1-5 minute granularity for responsive scaling (may be separate issue)

Open Questions

• Snapshot interval configuration - Per-resource? System-wide? What's the cost/latency tradeoff?
• Fair share algorithm - When multiple users are active, how is available capacity distributed? (Proportional to base? Equal split? TBD)
• Utilization smoothing - Use instantaneous value or rolling average to avoid oscillation?
• Cascade interaction - How does adaptive scaling work with hierarchical limits? If a parent entity has adaptive limits, do children inherit the scaled effective limit? If both parent and child are adaptive, which utilization metric applies?

Related Work

• #222 - Time-based dynamic rate limits (complementary approach)
• Centralized Configuration (ADR-001) provides resource-level storage patterns
• Usage snapshots provide the utilization data
• Part of the Dynamic Rate Limits initiative

Start with the set_resource_info() and set_limits() entry points, then read the usage snapshots and GSI2 resource-usage paths. Review how cascade mode and the CLI expose limits before resolving the open questions around snapshots, fair sharing, smoothing, and inheritance. Done means the success criteria are implemented and covered across resource metadata, adaptive limits, CLI output, and hierarchical limits.