Surface config skew on the /fleet page
- Dominant language
- Python
- Stars
- 1
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
`check_config_skew` polls LiteLLM's `/model/info` every `skew_interval_s` and
compares what the proxy reports against what the fleet expects. It populates
`srv.skew` and logs a warning. Nothing displays it.
That is why "LiteLLM routes have no api_base" went unnoticed: the system detected the route mismatch
correctly, wrote it to `srv.skew`, and the one page that would have shown it
has been returning a 500 ("AuthzError re-renders the page template").
The `/fleet` template already has a Skew column. It needs the page to render
and the value to be visible — a red row saying "missing from LiteLLM" turns a
three-layer trace into a five-second diagnosis.
**Three health signals, deliberately different in cost — worth labelling them
in the UI so nobody conflates them:**
| probe | interval | cost |
|---|---|---|
| vLLM `/health` on the box | `health_interval_s` (45s) | free, no API key |
| LiteLLM `/model/info` | `skew_interval_s` (180s) | free, metadata only |
| LiteLLM `/health` | `litellm_health_interval_s` (4h) | **runs real completions** |
The last one is an end-to-end test that consumes tokens, which is why it is
infrequent and is not the boot probe. At four hours it should be noise, but it
is worth watching rather than assuming.
Note the `extra` branch of the skew check (`bases - fleet_bases`) is vacuous
when no route has an `api_base` at all — it only catches LiteLLM having routes
the fleet does not know about.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.