atomantic / atomantic/PortOS

Compose AI runs from enabled Harness × Method × Service × Model × Effort instead of one provider record per combination

Open
#7,561 0 comments 0 reactions 0 assignees View on GitHub
decomposed effort:xhigh enhancement epic model:heavy planner:fable-5-1
Dominant language
JavaScript
Stars
38
Forks
31
Avg merge
22m
Merged PRs (30d)
988

Description

## Problem / Goal

Today an executable AI route is one flat provider record per **combination** of program, execution mode, and backend. `server/lib/aiToolkit/defaults/providers.sample.json` ships 55 of them: `claude-code`, `claude-code-tui`, `claude-code-bedrock`, `claude-code-tui-bedrock`, `claude-ollama`, `claude-ollama-tui`, `opencode-{ollama,lmstudio,mtplx,llama,vllm,sglang,orcarouter,openrouter,nvidia-nim,zen}{,-tui}`, `codex-{ollama,lmstudio}`, and so on. Holding an NVIDIA API key does not make "Pi TUI on NVIDIA", "Direct API on NVIDIA", or "OpenCode CLI on NVIDIA" available; each needs its own hand-made record carrying a duplicated endpoint, key, env block, and model list. Model catalogs are duplicated per record too (`nvidia-nim` and `nvidia-kimi` sit on one endpoint with two unrelated lists; `opencode-zen` and `opencode-zen-cli` are one service with two disjoint catalogs), and free-vs-paid is inferred from id regexes in `server/lib/modelPricing.js:263-352`.

Goal: a run is composed from five independently enabled axes — **Harness → Method → Service → Model → Effort** — and any compatible combination of enabled parts is available without a pre-made record. Model catalogs and plan (free / paid / subscription / local) live on the service. Any CLI/TUI combination can additionally run with or without a **credential bootstrap** wrapper (the ` run -- …` prefix `server/lib/credentialBootstrap.js` already supports per record). Named combinations are **presets** (what provider records are today), picked from one dropdown everywhere; a custom combination stays one "Custom…" step away and never adds a wall of selects to the ~99 client surfaces that pick a provider/model/effort today.

This is the tracking epic. It records the design decisions; the six child issues below deliver them in dependency order.

## Context

The connection graph shipped in September 2026 (#6359 → #6366–#6369, design record `docs/plans/2026-09-06-provider-connections-and-harnesses.md`) already separates three of the five axes as a **management overlay** over the flat records:

- `ai_connections` (`server/lib/db/schema/aiGraph.js:25`) — kind, transports `{protocol: {baseUrl}}`, credentials, `catalog {state: unknown|known|failed, models[]}`. Kinds unify local runtimes, gateways and bare API endpoints (`CREATABLE_CONNECTION_KINDS`, `server/lib/providerRouteRecipes.js:49`).
- `ai_harness_bindings` — connection × harness × variant, `enabled`, `selectedModels`.
- `ai_route_bindings` — one row per executable `provider_id`, `mode cli|tui|api`, `modelMap`, plus projection snapshots so `data/providers.json` stays the fully materialized execution contract for older releases (`server/lib/providerGraphRecords.js`, `planGraphReconciliation`).
- `PROVIDER_HARNESSES` (`server/lib/providerHarnesses.js:114`) — 10 rows (claude, opencode, kilo, openchamber, codex, antigravity, cursor, grok, kimi, pi) with `modes`, `protocol`, and a command `recipe` for **only claude, opencode, codex**. Pi, antigravity, cursor, grok and kimi are `protocol: 'native'` / `recipe: null` ("reaches only its own vendor service"), which is wrong for Pi: the binary accepts `--provider `, `--model /`, `--api-key`, per-provider env keys and custom OpenAI-compatible providers, so it is exactly the harness the motivating example needs.
- `PROVIDER_GATEWAYS` (`server/lib/providerGateways.js:57`): orcarouter, openrouter, nvidia-nim (OpenCode-only, key on a sibling `api` record). `LOCAL_RUNTIMES` (`server/lib/localProviderRuntime.js:71`): llama, ollama, lmstudio, mtplx, vllm, sglang (+ slotstream by id). `PROVIDER_VENDORS` (`server/lib/providerVendors.js:858`) is argv-shaped per binary. `credentialRegistry.js` has 22 credential rows with `tier: free|metered|none` but no link to provider records.

What the graph did **not** do, and this epic does:

1. Records are still authoritative; the graph reconciles against them. A combination with no record does not exist.
2. Availability requires a binding row plus a route row per mode. Enabling a harness and a service does not by itself offer the pair.
3. `provider.models[]` is still the catalog of record; the connection catalog is the union of its routes' alias keys (`connectionCatalogModels`, `server/lib/providerModelAliases.js:137`).
4. No plan/tier concept: entitlement is regex-inferred (`isFreeProvider`, `isFreeModelId`) and NVIDIA's free tier is not machine-readable at all (verified: `/v1/models` carries only `id/object/created/owned_by`), so it has to be user-declared.
5. The management UI is a drawer off the flat card list (`client/src/components/providers/ProviderConnections.jsx`); the page's mental model is still "one card per record, grouped by readiness".

Selection contract everywhere is the string triple `{providerId, model, effort}`: 47 persisted server-side selection sites (24 in `server/services/aiAssignments.js`, plus CoS task metadata, orchestration profile roles, task templates, decks, FableLoom, Creative Director, quota burn, Persistent Mind, POST, mood boards, catalog ingest, …) and ~99 client surfaces (63 files import `ProviderModelSelector`, 50 import `useProviderModels`). `metadata.provider/model/effort` cross the peer wire verbatim inside CoS task payloads (`peerCosSync.js:365`), with no provider entry in `schemaVersions.js`. Rewriting that contract is the option the 2026-09-06 design record rejected as "fleet-wide selection rewrite", and this epic keeps rejecting it.

Execution today (`server/services/promptRunner.js:1637`) branches on `provider.type`; CLI/TUI receive model + effort only through a cloned record (`{...provider, defaultModel, effort}`), and every argv/env/config mechanism is already table-driven per harness (`buildVendorCliArgs`, `buildEffortArgs`, `prefixOpencodeModel`, `buildCodexOssArgs`, `resolveClaudeCliModel`). Effort ladders are per harness and, for codex/antigravity, per model (`effortLevelsForProvider`, `server/lib/providerModels.js:527`).

## Decisions

Adjustments to the request as phrased are called out inline.

**D1 — Five axes; Method is constrained by Harness; "API" is a harness+method, not a harness peer of Claude.** Harness = the program that drives the model. PortOS's own HTTP runner (today `type: 'api'`, `executeApiRun`) is registered as harness `direct` (label "Direct API") with exactly one method, `api`. Every other harness declares its methods from `ROUTE_MODES` (`cli` headless, `tui` PTY): openchamber `cli` only, the rest `cli`+`tui`. Existing `ai_harness_bindings.harness_id = NULL` rows continue to mean `direct` through one application-layer normalizer (`normalizeHarnessId(null) === 'direct'`); the column stays nullable and the partial unique index `uq_ai_harness_bindings_api_variant` (`aiGraph.js:57`) is untouched — no row rewrite, no index migration.

**D2 — Local runtimes are Services.** Confirmed as the request suspected. A Service is any backend a harness can be pointed at. Three families, one shape:
- *subscription* — auth lives inside the harness (claude.ai login, ChatGPT/Codex login, Google for Antigravity, xAI for Grok Build, Cursor, Kimi). PortOS holds no key; readiness comes from `credentialInventory` source `cli`.
- *api-key* — Anthropic, OpenAI, Google, xAI, Bedrock, NVIDIA NIM, OpenRouter, OrcaRouter, OpenCode Zen, Cerebras, and a generic `openai-compatible` endpoint.
- *local runtime* — Ollama, LM Studio, MTPLX, llama.cpp, vLLM, SGLang, Slotstream (daemon readiness, install/launch recipes from `localRuntimeSetup.js`), plus `fleet-host` (a peer's OpenAI-compatible gateway).

**D3 — Service instance = definition + plan + credential + catalog.** Definitions are code (`SERVICE_DEFINITIONS`, absorbing `PROVIDER_GATEWAYS`, `LOCAL_RUNTIMES`, the AI rows of `credentialRegistry.js`, and new subscription rows). Instances are `ai_connections` rows with additive columns `slug` (unique, `[a-z0-9][a-z0-9-]*`, defaults to the definition id), `definition_id`, `plan` (`free|paid|subscription|local`), `enabled`. Two instances of one definition with different plans are legitimate and are how "NVIDIA free vs paid" and "OpenCode Zen free vs paid" are represented: each holds its own key and its own catalog. `plan` is **user-declared**; a definition may ship a per-plan catalog filter only where a real signal exists (OpenRouter `*:free`, OpenCode Zen `*-free`). `modelPricing.js` stays a leaf that accepts bare id strings (usage rows outlive provider configs, `:261-262`), so it never imports the service store: materialized records carry an additive, non-secret `servicePlan` field, `isFreeProvider(record)` reads it when present, and the id/endpoint regexes remain the fallback for string arguments and legacy presets.

**D4 — The catalog belongs to the service instance.** `ai_connections.catalog` becomes the source of truth, refreshed by the definition's listing strategy: `probe` (OpenAI `/v1/models`), `daemon` (Ollama `/api/tags`, `lms`, `mtplx models --json`), `harness` (`agy models`, `codex models`, `pi --list-models`, cursor), `static` (Anthropic). The null-vs-empty rule stays (`nextConnectionCatalog`: a failed probe keeps the last list). Harness-side naming is a pure adapter of (harness, service, canonical model) — today's `toExecutableModelName` — so `ollama/llama3` for OpenCode, `openai/gpt-…` for Pi, and the Bedrock id mapping for Claude are derived, never stored per record. Preset `models[]` becomes a materialized projection of the service catalog (∩ optional preset narrowing) for downgrade compatibility.

**D5 — Harness capability declarations replace `protocol: 'native'`.** Each `PROVIDER_HARNESSES` row declares `bindings[]` describing how it can be pointed at a service: `{ via: 'subscription', service: 'claude-subscription' }`; `{ protocol: 'anthropic', baseUrl: env ANTHROPIC_BASE_URL, credential: env ANTHROPIC_AUTH_TOKEN (required) }`; `{ service: 'bedrock', env: CLAUDE_CODE_USE_BEDROCK + AWS_BEARER_TOKEN_BEDROCK }`; `{ protocol: 'openai', via: opencodeConfig namespace + _ENV }`; `{ localRuntime: [ollama, lmstudio], via: --oss --local-provider }`; `{ protocol: 'openai'|'anthropic', via: pi --provider + --model provider/id + env key, custom endpoints through Pi's models.json }`; `direct`: `{ protocol: 'openai', via: endpoint + Bearer apiKey }`. `compatible(harness, service)` = some binding is satisfied by the service's definition id, transports, or runtime kind. Antigravity, Grok Build, Cursor and Kimi declare a single subscription binding; Kilo and OpenChamber stay classifiable but not composable, exactly as today.

**D6 — The selection contract does not change.** A `providerId` is a **preset id** (today's record id, byte-for-byte) or a **composite id** `.@` — e.g. `pi.tui@nvidia-nim-free`, `direct.api@ollama`, `opencode.cli@openrouter`. `.` and `@` are outside the record id charset (`^[a-z0-9][a-z0-9-]*$`, `aiToolkit/validation.js:75`), so a composite can never collide with a record and is never stored as one. A shared `providerRefSchema` accepts both grammars wherever a **selection** is validated — including the toolkit `runSchema.provider` (`aiToolkit/validation.js:284`) — while `providerSchema.id` (stored records, `:75`) stays as is; a materialized composite is an in-memory record that is never passed through `providerSchema`. The preset-only guards are the existing record-id schemas left unchanged and tested: `providerActiveSchema` (`:280`), `providerSchema.fallbackProvider`, and `taskTypeOverrides` (`lib/validation.js:281`). The toolkit's `getProvider(id)` gains a host-injected composite resolver (same injection seam as the graph store) that materializes an in-memory record from the harness recipe, the service instance and defaults, so `promptRunner`, CoS dispatch, fallbacks, Shell launch and public review execute a composite through the code paths they use today. `activeProvider`, `fallbackProvider` and `taskTypeOverrides` remain **preset-only** (a downgraded release must still resolve its default); "Use as default" on a composite saves a preset first.

**D7 — Presets are today's provider records, made structural.** Additive record fields `harnessId`, `method` (mirrors `type`), `serviceId`. Connection-owned fields (endpoint, apiKey, envVars, secretEnvVars, `*Backed`/`gatewayBacked` markers, models) are derived from the service on every save and materialized into `data/providers.json` through the existing projection (`withConnectionOwnedFields`, `buildRouteRecord`), so every release that runs today's file keeps executing presets. Records the classifier cannot map (custom command, dynamic env, external config — `CONNECTION_ISOLATION_REASONS`) stay **legacy presets**: selectable, executable, editable, just not derived. `ai_route_bindings` keeps its projection-recovery role.

**D8 — Enablement is per axis; a pair needs no row to exist.** Harness enabled = detected on PATH (`providerRuntimeInstaller`) or explicitly enabled, minus an explicit off toggle stored in settings (`settings.harnesses[id].enabled`; `direct` always on). Service enabled = instance `enabled` ∧ credential present (or subscription login detected) ∧ for local runtimes not hard-down (standby counts, per `standbyWhenStopped`). A combination is **offered** when both are enabled, compatible (D5), and the caller mode policy (`server/lib/callerModePolicy.js`) allows the method. `ai_harness_bindings` become optional narrowing (label, `selectedModels`) rather than a prerequisite.

**D9 — Effort ladder keyed by (harness, model).** Same ladders and the same emission table; `effortLevelsForProvider` consults `harnessId` first and command sniffing second. The catalog endpoint publishes `effortLevels` per harness and `effortLevelsByModel` where model-gated (codex, antigravity), so the browser stops re-deriving ladders from a sanitized record.

**D10 — Preset-first single dropdown; compose behind "Custom…".** `ProviderModelSelector` lists enabled presets grouped by harness, then one "Custom combination…" entry opening a compact compose popover (harness → method → service → model → effort) that emits a composite id and offers "Save as preset". Unavailable saved values stay visible with a reason (existing rule). The AI Providers page becomes three tabs — **Harnesses**, **Services**, **Presets** — at `/ai/harnesses`, `/ai/services/:serviceId`, `/ai/presets/:presetId`; `/ai/edit/:providerId` stays an alias of the preset editor; `/ai/connections*` redirects to Services; all registered in `NAV_COMMANDS`.

**D11 — Credential bootstrap is an optional launch wrapper, independent of harness and service.** Today `credentialBootstrap {setupCommand?, command, args?, harnessId?, argsSeparator?}` is a per-record, connection-owned field (`aiToolkit/validation.js:205`, `MODE_GROUPED_KEYS`) and every spawn site routes through `resolveCliSpawn`. It becomes a small registry of configured **bootstrap apps** in settings (`settings.credentialBootstraps[slug] = {label, command, args, argsSeparator?, setupCommand?, harnessNames: {[harnessId]: string}}`, the per-harness name map replacing today's single `harnessId`). A composite may carry one as an optional suffix — `pi.tui@nvidia-nim+corp-auth` — and a preset stores `credentialBootstrapId`; materialization writes the inline `credentialBootstrap` object exactly as today so `resolveCliSpawn`, the public-review skip, process-group teardown (#7496) and older releases are untouched. Applies to `cli`/`tui` only. A service instance may declare its credential as `via: 'bootstrap'` (no stored key; supplied at spawn), in which case a combination on it **requires** a bootstrap; a service with a stored key runs with or without one.

**D12 — No federation wire change.** Composite ids ride task metadata as strings exactly as preset ids do; a receiver resolves them against its own services or shows an unresolved pin. Services, credentials, harness enablement and catalogs never federate (ADR `docs/decisions/2026-08-08-privacy-records-machine-local.md`). No `schemaVersions.js` bump.

## Slices (children)

Dependency order; each child carries its own acceptance criteria and dispatch labels.

- [x] #7562 — Registries: harness capability bindings, `direct` harness, service definitions, compatibility predicate, `materializeRoute`
- [x] #7563 — Service instances: slug/plan/enabled columns, per-definition catalog listing and plan filters, `/api/providers/services`
- [ ] #7564 — Harness enablement + composite id resolution through the toolkit and every run path; `/api/providers/catalog`
- [ ] #7565 — Presets: structural record fields, additive migration, service-derived materialization, generated samples
- [ ] #7566 — Selectors: preset-first `ProviderModelSelector` with compose popover; harness-keyed effort; bespoke selects migrated
- [ ] #7567 — AI Providers page: Harnesses / Services / Presets tabs, routes, nav manifest, docs

## Acceptance criteria (epic-level)

- [ ] With an NVIDIA NIM service instance holding a key and the `pi` binary installed, `pi.tui@nvidia-nim`, `pi.cli@nvidia-nim`, `direct.api@nvidia-nim` and `opencode.cli@nvidia-nim` are offered in the selector and execute end to end with no record in `data/providers.json`.
- [ ] Two service instances of one definition with different plans (e.g. `opencode-zen` free and paid) expose different catalogs and attribute cost differently on the Usage page.
- [ ] Every one of the 55 shipped sample records resolves to a preset with `harnessId/method/serviceId` or is reported as a legacy preset with a `CONNECTION_ISOLATION_REASONS` code; none is dropped, renamed, or has its args/env/models/pins/consent changed by the migration.
- [ ] Every persisted `{providerId, model, effort}` selection keeps working unchanged; a saved composite id survives a downgrade as a visible unresolved pin, never a silent substitution.
- [ ] `data/providers.json` written by the new release is executable by the previous release (fixture test against the prior parser), and `activeProvider` is always a preset id.
- [ ] With one bootstrap app configured, `claude.cli@anthropic+corp-auth` spawns ` run claude-code -- …` (per the app's harness name map) and `claude.cli@anthropic` spawns `claude` directly; a service declared `via: 'bootstrap'` refuses the unsuffixed composite with a reason.
- [ ] The selector on any surface shows one dropdown by default; the compose flow is keyboard-operable and mobile-usable per `client/src/AGENTS.md`.
- [ ] No provider call is made at boot, on enabling a harness or service, or on catalog display; catalog refresh is an explicit action.

## Out of scope

- Rewriting persisted selections to a structured `{harness, method, service}` object (rejected in the 2026-09-06 design record and again here).
- Making Kilo or OpenChamber composable; new harnesses; new vendors beyond the definitions listed in D2.
- Credential encryption at rest; federating services or catalogs; a peer-wire schema bump.
- Auto-detecting NVIDIA free-tier models (not machine-readable; plan is declared).
- Changing effort ladders or the per-harness argv/config emission.

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing graph and execution entry points named in server/lib/db/schema/aiGraph.js, server/lib/providerHarnesses.js, server/lib/providerGraphRecords.js, and server/services/promptRunner.js. Review the six dependency-ordered child issues before choosing work; done means the selected child’s design is implemented while preserving the existing provider/model/effort selection contract and legacy projections.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, react
Domain
ai, backend-api-design, frontend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.