ContextLab / ContextLab/llmXive
spec 026: requirement-driven model selection consolidation
- Dominant language
- Python
- Stars
- 4
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
## What
Consolidate every model-selection site in the platform into ONE requirement-driven mechanism with a strict boundary between **choosing** a model chain and **executing** it:
```
snap = runtime_snapshot(...) # acquisition — networked, fallible
decision = select(requirement, snap) # selection — pure, deterministic
response = execute(decision.chain, messages, options=..., backend_factory=...)
```
Call sites express **intent** (a named requirement profile), never a model string. Spec:
[`specs/026-model-selection-consolidation/spec.md`](specs/026-model-selection-consolidation/spec.md) ·
plan: [`plan.md`](specs/026-model-selection-consolidation/plan.md) · branch `026-model-selection-consolidation`.
## Relationship to #1242 (traceability)
This issue exists because spec 026 does **NOT** implement #1242's acceptance criteria. #1242
("self-improvement through lessons learned") asks for two cron-driven self-improvement lanes.
Spec 026 is the **model-selection prerequisite** for spec 025's lane, split off as its own spec + PR
by maintainer decision 2026-07-16 (recorded at `spec.md:6`) so it can land first. Work on this branch
should be judged against the criteria below, not against #1242's.
## Acceptance (from spec.md)
- **FR-001/FR-008**: `select()` is a pure function of `(requirement, snapshot, policy_version)`
returning a byte-identical ordered chain; `execute()` never re-ranks.
- **FR-006**: eligibility = versioned capability policy ∩ live snapshot; the catalog proves
presence/price, never competence; unknown cost is never free.
- **FR-010/FR-011/FR-016**: existing router safeguards preserved; paid permission checked twice;
paid calls reserve against the strict-CAS ledger on `refs/llmxive/credit-ledger`.
- **FR-012/FR-013**: all call sites express one of the 6 named profiles; registry agents, the router,
the execution tier-ladder, the catalog filter and every direct pin resolve through the mechanism.
- **FR-014**: a static enforcement test rejects production model-id literals outside the canonical
layer; `DEFAULT_MODEL` + `MODEL_FALLBACKS` are deleted.
- **SC-007**: behavior preserved against a fixed migration snapshot (`qwen.qwen3.5-122b` stays the
default; gemma-3→gemma-4 is the one maintainer-approved correction).
- **SC-008**: real snapshot assembly + real calls across the free tiers, the vision tier, and the
local fallback.
## Status
Slices 1–6 complete (contracts, snapshot+LKG, router primitive, ledger, pure `select`, `execute`,
independence, routing chokepoint, execution-tier migration). Slice 7 (compatibility consumers +
`AgentRegistryEntry` schema migration + US5 AC1/AC4) in design. Slice 8 (static enforcement +
deletions) last.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with specs/026-model-selection-consolidation/spec.md, then read plan.md and the Slice 7 and Slice 8 status. Trace the named profiles, compatibility consumers, AgentRegistryEntry migration, static enforcement, and deletions against the listed acceptance criteria. Done means all model-selection sites use the mechanism, safeguards remain intact, and the migration behavior and real-call criteria pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100