Layr-Labs / Layr-Labs/d-inference
[bug] `enabled_models` is not read from provider.toml; `status`/`doctor` report a model filter that cannot be set
- Dominant language
- Go
- Stars
- 522
- Forks
- 113
- Avg merge
- 17h 26m
- Merged PRs (30d)
- 111
Description
### Component
coordinator (Go)
### What happened?
# `enabled_models` is not read from provider.toml; `status`/`doctor` report a model filter that cannot be set
**Version:** darkbloom 0.8.14
**Hardware:** Apple M5 Pro, 48 GB RAM, 20 GPU cores, macOS 26.6.1 (Tahoe)
**Model served:** `qwen3.6-35b-a3b-vl-mtp-mxfp8`
## Summary
`doctor` fails a `model fits in RAM` check and tells me to fix it by setting
`enabled_models` in `provider.toml`. That key appears to have no effect — the
provider's model selection actually lives in the launchd plist, and `status`
continues to report `auto-select` / `filter: none` regardless of what the TOML
contains. There are also three numbers for "how much memory is available" that
disagree with each other.
## 1. `enabled_models` in provider.toml has no effect
`doctor` output:
```
[FAIL] model fits in RAM — qwen3.6-35b-a3b-vl-mtp-mxfp8 needs ~30.3 GB but only
23.0 GB is usable — it will show online but every request fails to load.
↳ fix: set `enabled_models` in provider.toml to a model that fits: ...
```
I tried all of these, restarting the provider after each:
```toml
# A — under [provider]
[provider]
enabled_models = ["qwen3.6-35b-a3b-vl-mtp-mxfp8"]
# B — top level
enabled_models = ["qwen3.6-35b-a3b-vl-mtp-mxfp8"]
# C — alternate key name seen in the binary
[provider]
models = ["qwen3.6-35b-a3b-vl-mtp-mxfp8"]
```
After each: `darkbloom restart` then `darkbloom status` →
```
Configured model: auto-select
Enabled model filter: none
```
Control: `idle_timeout_mins = 60` set in the same `[provider]` section **is**
picked up (`Idle timeout: 60m`), so the section is being parsed — it's
specifically `enabled_models` that isn't honoured.
`strings` on the binary shows `enabled_models`, `enabledModels` and `models` as
known keys, plus the help text `"Show every discovered local model, ignoring the
config enabled_models filter."` — so the filter exists somewhere, but I can't find
a TOML form that sets it.
## 2. The real selection lives in the launchd plist, and `status` doesn't reflect it
`~/Library/LaunchAgents/io.darkbloom.provider.plist`:
```
ProgramArguments:
darkbloom start --foreground
--coordinator-url wss://api.darkbloom.dev/ws/provider
--model qwen3.6-35b-a3b-vl-mtp-mxfp8
--idle-timeout 60
```
So the daemon *is* pinned to one model — but `status` still says
`Configured model: auto-select` and `Enabled model filter: none`. If `status` is
reporting the TOML rather than the effective launchd selection, that's misleading;
it made me think my config was being ignored when the daemon was actually correct.
`darkbloom start --model ` also does not write the selection back into
`provider.toml` — it only updates the plist.
## 3. `doctor` scans every local model, not the one being served
The FAIL initially named `mlx-community/Qwen3.8-27B-8bit` — a model the daemon
would never load, and which `darkbloom models` itself lists under *"no longer
served by the network"*. Removing it just moved the FAIL to the next largest local
model, and so on:
```
mlx-community/Qwen3.8-27B-8bit 33.0 GB → removed → FAIL moves on
lmstudio-community/Qwen3.6-35B-A3B-MLX-6bit 32.5 GB → removed → FAIL moves on
qwen3.6-35b-a3b-vl-mtp-mxfp8 → now flagged
```
If a `--model` selection is in force, should the RAM check evaluate only the
selected model(s)? As it stands the check reports "every request fails to load"
about models that will never be requested.
## 4. Three disagreeing memory figures
Same box, same moment, machine otherwise idle (90% memory free, 3.1 GB wired):
| source | value |
|---|---|
| `darkbloom models` catalog | `~21.3 GB (≥ 32 GB RAM)` |
| `darkbloom status` | `Inference memory: 44 GB available` |
| `darkbloom doctor` | `needs ~30.3 GB but only 23.0 GB is usable` |
The `doctor` "usable" figure also drifted between consecutive runs in one session
(28.1 → 27.9 → 27.8 → 23.0 GB) with no meaningful change in system load. And the
catalog's `~21.3 GB` vs doctor's `~30.3 GB` for the same model is a 9 GB spread —
enough to flip a pass/fail on a 48 GB machine.
## What I'd expect
1. `enabled_models` in `provider.toml` either works, or `doctor`'s suggested fix
points at whatever the real mechanism is (`darkbloom start --model ...`?).
2. `status` reports the *effective* selection (the plist args), not just the TOML.
3. The RAM check evaluates the selected model(s) when a selection is in force.
4. The catalog size, `status` available-memory, and `doctor` usable-memory figures
are reconciled, or the doc explains what each one measures.
## Environment / other checks
Everything else passes: SIP enabled, hardened runtime, authenticated root, MDM
profile installed, coordinator trust `hardware / online`, binary hash verified,
Secure Enclave attestation active, version up to date.
Happy to run anything that would help narrow this down.
### Expected behavior
we should be able to select the model from the catalog
### Steps to reproduce
install Darkbloom in a system with multiple HF models and run Darkbloom
### Environment
Mac 26.6.1
### Logs
```shell
```
### Confirmations
- [x] I searched existing issues and this isn't a duplicate
Contributor guide
Research direction
Start with the provider.toml parsing and the ~/Library/LaunchAgents/io.darkbloom.provider.plist generated by `darkbloom start --model`. Compare the configured and effective model selections reported by `status`, then inspect how `doctor` chooses models and calculates usable memory. Done means model selection works or its documented mechanism is reported consistently, and the RAM check and memory figures agree with that selection.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100