Layr-Labs / Layr-Labs/d-inference
doctor/verify: add a routability check (node-scoped /v1/models + self-route completion) — every local check passes during the never-advertised wedge
- Dominant language
- Go
- Stars
- 522
- Forks
- 113
- Avg merge
- 17h 26m
- Merged PRs (30d)
- 111
Description
## Problem
`darkbloom doctor` and `darkbloom verify` only look at the box: hardware, MDM profile, binary hash, trust level as the daemon reports it, "model fits in RAM", coordinator HTTP health. During the restart→never-advertised wedge (#692, datapoints 4–6; Report ID 5523 for tonight's 0.9.2 case) all of those PASS for the whole session:
- trust `hardware / online`, reason `continuity`
- model warm, MTP active, `advertised_models` in daemon-state = the model
- attestation challenges answered
while the coordinator's view is the opposite:
- node-scoped `GET /v1/models` with the provider-bound customer key → `[]`
- `POST /v1/chat/completions` with `X-Darkbloom-Route: self` → `503 model_not_loaded` ("not available on your machine — load it on your node and retry")
- 0 routed requests
So an operator running `doctor` on a node that has earned nothing for an hour is told everything is fine. The two calls above are the only signal that was red, and they are cheap and already public.
## Ask
Add a "ROUTABILITY (does the coordinator route to this node?)" section to `doctor` (and the same check to `verify`):
1. node-scoped advertised set (the customer-key `/v1/models`) — PASS when it contains the warm models, WARN with the mismatch when it is empty or missing one (this is the #692 shape, and also the 0.7.x "attested but nothing preloaded" shape).
2. one self-route completion (`max_tokens: 8`) — PASS on a completion, FAIL with the coordinator's error code verbatim (`model_not_loaded` / `model_unavailable` / `rate_limit_exceeded`) so the operator can tell "coordinator dropped my load report" from "the box is not serving".
3. Print the session's registration time and the time of the last routed request beside them.
`status` could carry a one-line version ("coordinator advertises: gemma-4-26b" / "coordinator advertises: NONE — see doctor").
## Context
Operator-side we now run these two calls every 15 min ourselves (they are how the datapoints on #692 were measured); it would be better if the vendor CLI did it. Related: #766 (`enabled_models` reporting), #688 (attested node gets 0 requests).
Contributor guide
Research direction
Start by locating the implementations of darkbloom doctor, darkbloom verify, and status. Reproduce the node-scoped GET /v1/models and self-route POST /v1/chat/completions checks with the customer key, then trace how session registration and last-routed-request times are available. Done means both commands show routability results, coordinator error codes, and the requested timing details.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- api, cli
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100