EpistasisLab / EpistasisLab/motoro

Let callers supply discovered model capabilities instead of re-resolving from _REGISTRY

Open
#2 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
0
Forks
0
Avg merge
2m
Merged PRs (30d)
3

Description

## Problem

`_sampling_kwargs()` (`agentic_core/services/llm_service.py:155-169`) is the only place that decides what sampling control goes on the wire, and it re-resolves capabilities from the model **string**:

```python
caps = get_capabilities(model_str)
if caps.supports_temperature:
return {"temperature": config.temperature}
if caps.supports_effort and config.effort:
return {"reasoning_effort": config.effort}
return {}
```

`ModelConfig` (`agentic_core/schemas/agent.py:31`) has no field for capabilities, so a caller that has already determined a model's real capabilities has no way to pass them in. Core looks the model up in `_REGISTRY` again and ignores what the caller knew.

`_REGISTRY` currently holds five entries — `claude-opus-5`, `claude-opus-4-8`, `claude-opus-4-7`, `claude-fable-5`, `claude-sonnet-5` — and everything else falls through to `DEFAULT_CAPABILITIES` (`supports_temperature=True`). Substring matching covers point releases (`claude-opus-5-1` matches `claude-opus-5`) but not new families.

## Why this now produces wrong wire calls

ASAREE reads Anthropic's `GET /v1/models`, which returns a real per-model capability tree (`effort.supported` plus a `supported` flag per level). The live ladders are genuinely more granular than the registry's single hardcoded list:

| Model | Effort levels (live API) |
|---|---|
| 5-series, opus-4-7, opus-4-8 | low / medium / high / xhigh / max |
| sonnet-4-6, opus-4-6 | low / medium / high / max (no xhigh) |
| opus-4-5 | low / medium / high |
| haiku-4-5, sonnet-4-5 | `effort.supported: false` |

For any model outside those five registry entries — `claude-opus-4-6`, `claude-sonnet-4-6`, and every model Anthropic ships from here on — the UI now shows an Effort control sourced from the live API, the user picks a level, and core sends `temperature` instead. The effort choice is silently dropped, or the call 400s if the model genuinely rejects temperature.

The same wall blocks live OpenAI discovery. OpenAI's `GET /v1/models` returns no capability data at all (`id`/`object`/`created`/`owned_by`/`shutdown_date` only, confirmed by live probe), so a consumer has to maintain its own table for the `gpt-5.x` / `o*` lines. With no way to hand that table to core, the picker would offer `reasoning_effort` while core sent `temperature` — so the work can't be done consumer-side today.

## Proposed change

Add an optional capabilities override to `ModelConfig` and have `_sampling_kwargs()` prefer it, falling back to `get_capabilities(model_str)` when it's absent:

```python
caps = config.capabilities or get_capabilities(model_str)
```

Backward compatible — nothing that omits the field changes behaviour — and `model_capabilities` stays the default oracle for callers that have nothing better.

## Alternative considered

Keep adding `_REGISTRY` entries for each new model family. Rejected: it means re-tagging agentic-core every time a provider ships a model, which is the treadmill live discovery exists to get off. It also can't work for OpenAI, where the registry would have to encode a request-mode rule (temperature is legal precisely when `reasoning_effort` is `none` on the 5.1+ families) that a per-model capability struct can't express.

## Notes

- The module docstring's warning that litellm's `get_supported_openai_params` is not a reliable oracle still holds — this issue is about letting a caller with a *better* oracle use it, not about trusting litellm.
- No cross-provider capability standard exists: Anthropic returns a full capability tree, Azure Foundry's `capabilities` field only ever holds `{"chat_completion": "true"}` (a modality, not a sampling contract), and OpenAI returns nothing. Azure would keep using `get_capabilities()` under this change.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with _sampling_kwargs() in agentic_core/services/llm_service.py:155-169 and ModelConfig in agentic_core/schemas/agent.py:31. Trace the existing capability type and current get_capabilities() fallback, then verify that a supplied override is preferred while configurations without one retain current behavior; done means caller-provided capabilities control the wire sampling parameter.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.