ADORSYS-GIS / ADORSYS-GIS/lightbridge-governance

[Ticket]: VS Code extension ignores 429 — a rate limit empties the model picker

Aperta
#232 1 commento 0 reazioni 1 assegnatario Rivendicata da @Guy-Ghis Vedi su GitHub
bug ticket
Lingua principale
Rust
Stelle
1
Fork
2
Merge medio
13h 13m
PR unite (30g)
110

Descrizione

## Summary

The VS Code extension has no handling for HTTP 429. A rate limit currently produces a hard error on the chat path and — worse — **empties the model picker** on the catalogue path.

## Current behaviour

`provider.ts` handles 401/403 explicitly and falls through for everything else:

```ts
if (res.status === 401 || res.status === 403) { throw ...NoPermissions... }
if (!res.ok) { throw new Error(`${redact(url)} returned ${res.status}.`); }
```

`catalogue.ts` throws on `!res.ok`, the provider catches it and returns `[]`:

```ts
// There is no cached-model fallback here on purpose: serving a stale
// catalogue after the gateway has stopped answering is how a model that
// policy has withdrawn stays selectable.
return [];
```

That reasoning is right for a gateway that has **stopped answering**. A 429 is the gateway answering clearly — "ask again later". Treating "throttled" as "you may not have these models" is the wrong branch, and the developer sees an empty picker.

## The house already has the pattern

`crates/governance-copilot/src/client.rs` retries 429/5xx/timeouts with bounded exponential backoff, **preferring the server's `Retry-After` over its own schedule**, with a test pinning that preference (`retry_delay_prefers_retry_after_header_over_backoff`). Port that shape rather than inventing one.

## Scope

- **Catalogue**: retry honouring `Retry-After`, bounded. If still throttled, serve the cached list **only while within `catalogueTtlMs`** — so a throttle does not blank the picker, but a genuinely withdrawn model still disappears within the TTL.
- **Chat**: retry only *before* the first part is reported to `progress`. Once text has been emitted the response is half-delivered and a retry would duplicate it, so a mid-stream 429 must surface as an error. This asymmetry needs to be explicit in the code — it is exactly what a later refactor "simplifies" into a bug.
- In-flight de-duplication on the catalogue already collapses a burst of picker queries into one request; no extra client-side queue is needed for that path.

## Explicitly out of scope: proactive throttling

The gateway advertises no limit. Measured on a 200 response from `api.ai.camer.digital`, the only headers are `server: nginx` and `x-envoy-upstream-service-time` — no `RateLimit-*`, no `Retry-After`. Pacing against an unknown budget means picking a number that either throttles developers who were nowhere near the limit or fails to prevent the 429 anyway.

The cheaper fix is at the other end: have the gateway emit `RateLimit-Limit` / `RateLimit-Remaining` / `RateLimit-Reset` (Envoy can produce these). Then the extension, Claude Code and Codex can all pace against a real budget. Worth a separate ticket against `ai-helm`.

Also worth establishing before either: whether the limit is per-user, per-tenant or per-model. A per-tenant limit means one developer's batch job throttles everyone, and no client-side pacing can fix that.

## AI Usage Declaration

AI-assisted. Header behaviour measured against the live gateway, not assumed. A human owns intent, verification and consequences.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.