ADORSYS-GIS / ADORSYS-GIS/lightbridge-governance
[Ticket]: VS Code extension ignores 429 — a rate limit empties the model picker
- Lingua principale
- Rust
- Stelle
- 1
- Fork
- 2
- Merge medio
- 13h 13m
- PR unite (30g)
- 110
Descrizione
## Summary
The VS Code extension has no handling for HTTP 429. A rate limit currently produces a hard error on the chat path and — worse — **empties the model picker** on the catalogue path.
## Current behaviour
`provider.ts` handles 401/403 explicitly and falls through for everything else:
```ts
if (res.status === 401 || res.status === 403) { throw ...NoPermissions... }
if (!res.ok) { throw new Error(`${redact(url)} returned ${res.status}.`); }
```
`catalogue.ts` throws on `!res.ok`, the provider catches it and returns `[]`:
```ts
// There is no cached-model fallback here on purpose: serving a stale
// catalogue after the gateway has stopped answering is how a model that
// policy has withdrawn stays selectable.
return [];
```
That reasoning is right for a gateway that has **stopped answering**. A 429 is the gateway answering clearly — "ask again later". Treating "throttled" as "you may not have these models" is the wrong branch, and the developer sees an empty picker.
## The house already has the pattern
`crates/governance-copilot/src/client.rs` retries 429/5xx/timeouts with bounded exponential backoff, **preferring the server's `Retry-After` over its own schedule**, with a test pinning that preference (`retry_delay_prefers_retry_after_header_over_backoff`). Port that shape rather than inventing one.
## Scope
- **Catalogue**: retry honouring `Retry-After`, bounded. If still throttled, serve the cached list **only while within `catalogueTtlMs`** — so a throttle does not blank the picker, but a genuinely withdrawn model still disappears within the TTL.
- **Chat**: retry only *before* the first part is reported to `progress`. Once text has been emitted the response is half-delivered and a retry would duplicate it, so a mid-stream 429 must surface as an error. This asymmetry needs to be explicit in the code — it is exactly what a later refactor "simplifies" into a bug.
- In-flight de-duplication on the catalogue already collapses a burst of picker queries into one request; no extra client-side queue is needed for that path.
## Explicitly out of scope: proactive throttling
The gateway advertises no limit. Measured on a 200 response from `api.ai.camer.digital`, the only headers are `server: nginx` and `x-envoy-upstream-service-time` — no `RateLimit-*`, no `Retry-After`. Pacing against an unknown budget means picking a number that either throttles developers who were nowhere near the limit or fails to prevent the 429 anyway.
The cheaper fix is at the other end: have the gateway emit `RateLimit-Limit` / `RateLimit-Remaining` / `RateLimit-Reset` (Envoy can produce these). Then the extension, Claude Code and Codex can all pace against a real budget. Worth a separate ticket against `ai-helm`.
Also worth establishing before either: whether the limit is per-user, per-tenant or per-model. A per-tenant limit means one developer's batch job throttles everyone, and no client-side pacing can fix that.
## AI Usage Declaration
AI-assisted. Header behaviour measured against the live gateway, not assumed. A human owns intent, verification and consequences.
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.