EpistasisLab / EpistasisLab/motoro
Let callers supply discovered model capabilities instead of re-resolving from _REGISTRY
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 2m
- Merged PRs (30d)
- 3
Description
## Problem
`_sampling_kwargs()` (`agentic_core/services/llm_service.py:155-169`) is the only place that decides what sampling control goes on the wire, and it re-resolves capabilities from the model **string**:
```python
caps = get_capabilities(model_str)
if caps.supports_temperature:
return {"temperature": config.temperature}
if caps.supports_effort and config.effort:
return {"reasoning_effort": config.effort}
return {}
```
`ModelConfig` (`agentic_core/schemas/agent.py:31`) has no field for capabilities, so a caller that has already determined a model's real capabilities has no way to pass them in. Core looks the model up in `_REGISTRY` again and ignores what the caller knew.
`_REGISTRY` currently holds five entries — `claude-opus-5`, `claude-opus-4-8`, `claude-opus-4-7`, `claude-fable-5`, `claude-sonnet-5` — and everything else falls through to `DEFAULT_CAPABILITIES` (`supports_temperature=True`). Substring matching covers point releases (`claude-opus-5-1` matches `claude-opus-5`) but not new families.
## Why this now produces wrong wire calls
ASAREE reads Anthropic's `GET /v1/models`, which returns a real per-model capability tree (`effort.supported` plus a `supported` flag per level). The live ladders are genuinely more granular than the registry's single hardcoded list:
| Model | Effort levels (live API) |
|---|---|
| 5-series, opus-4-7, opus-4-8 | low / medium / high / xhigh / max |
| sonnet-4-6, opus-4-6 | low / medium / high / max (no xhigh) |
| opus-4-5 | low / medium / high |
| haiku-4-5, sonnet-4-5 | `effort.supported: false` |
For any model outside those five registry entries — `claude-opus-4-6`, `claude-sonnet-4-6`, and every model Anthropic ships from here on — the UI now shows an Effort control sourced from the live API, the user picks a level, and core sends `temperature` instead. The effort choice is silently dropped, or the call 400s if the model genuinely rejects temperature.
The same wall blocks live OpenAI discovery. OpenAI's `GET /v1/models` returns no capability data at all (`id`/`object`/`created`/`owned_by`/`shutdown_date` only, confirmed by live probe), so a consumer has to maintain its own table for the `gpt-5.x` / `o*` lines. With no way to hand that table to core, the picker would offer `reasoning_effort` while core sent `temperature` — so the work can't be done consumer-side today.
## Proposed change
Add an optional capabilities override to `ModelConfig` and have `_sampling_kwargs()` prefer it, falling back to `get_capabilities(model_str)` when it's absent:
```python
caps = config.capabilities or get_capabilities(model_str)
```
Backward compatible — nothing that omits the field changes behaviour — and `model_capabilities` stays the default oracle for callers that have nothing better.
## Alternative considered
Keep adding `_REGISTRY` entries for each new model family. Rejected: it means re-tagging agentic-core every time a provider ships a model, which is the treadmill live discovery exists to get off. It also can't work for OpenAI, where the registry would have to encode a request-mode rule (temperature is legal precisely when `reasoning_effort` is `none` on the 5.1+ families) that a per-model capability struct can't express.
## Notes
- The module docstring's warning that litellm's `get_supported_openai_params` is not a reliable oracle still holds — this issue is about letting a caller with a *better* oracle use it, not about trusting litellm.
- No cross-provider capability standard exists: Anthropic returns a full capability tree, Azure Foundry's `capabilities` field only ever holds `{"chat_completion": "true"}` (a modality, not a sampling contract), and OpenAI returns nothing. Azure would keep using `get_capabilities()` under this change.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with _sampling_kwargs() in agentic_core/services/llm_service.py:155-169 and ModelConfig in agentic_core/schemas/agent.py:31. Trace the existing capability type and current get_capabilities() fallback, then verify that a supplied override is preferred while configurations without one retain current behavior; done means caller-provided capabilities control the wire sampling parameter.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100