KV cache column is empty on every host

/fleet shows — under KV cache for all three servers. The column exists to surface the measured cache against the served context window, so that a max_model_len above what the cache holds — which makes vLLM hang rather than error — is visible rather than being found by reading a startup log.

Restarting the portal did not populate it.

Cause

fetch_observed builds its URL from srv.api_base, which carries a /v1 suffix. So it requests /v1/metrics, where vLLM serves metrics at the root.

Confirmed on the llmao host against the 27B:

$ curl -sS http://103.196.86.105:17140/metrics | grep -c cache_config_info
3
$ curl -sS http://103.196.86.105:17140/v1/metrics | head -c 100
{"error":"Unauthorized"}

The non-200 is caught deliberately — fetch_observed degrades to None rather than breaking the probe loop, since vLLM's metric names are not a stable API. That is right in general and means this particular mistake fails silently, with nothing in the log.

Introduced when the scrape was written against an api_base that did not yet include /v1.

Fix

Strip the suffix before appending the metrics path:

python
root = base[:-3].rstrip("/") if base.endswith("/v1") else base.rstrip("/")
resp = await client.get(f"{root}/metrics", headers=headers)

/v1/models is unaffected — that endpoint genuinely is under /v1.

Ruled out

Both halves of the change are deployed; grep -c returns 2 for kv_cache in templates/fleet.ezt and 2 for parse_kv_cache_tokens in llmao/fleet.py. So this is not a partial apply.

The scrape firing only on the transition into SERVING is also not the cause — it retries whenever kv_cache_tokens is still None, which it always is here.

Start in llmao/fleet.py at fetch_observed and trace how srv.api_base is used for the metrics request; compare that with the confirmed root /metrics endpoint and the unaffected /v1/models endpoint. Check templates/fleet.ezt for the displayed KV-cache field, then verify that /fleet shows measured values for the servers instead of em dashes and that the probe loop still tolerates unavailable metrics.

Written by the indexing model from the issue text.