atomantic / atomantic/PortOS

Resolve composite provider ids (harness.method@service[+bootstrap]) through the toolkit and every run path, with per-axis harness enablement

Open
#7,564 2 comments 0 reactions 1 assignee Claimed by @atomantic View on GitHub
effort:xhigh enhancement in-progress model:heavy plan planner:fable-5-1
Dominant language
JavaScript
Stars
38
Forks
31
Avg merge
22m
Merged PRs (30d)
988

Description

Part of #7561 (epic).
Blocked by #7562, #7563

## Problem / Goal

A combination of an enabled harness and an enabled service is not runnable unless a provider record exists for it. Deliver per-axis enablement and the **composite id** `.@[+]` (e.g. `pi.tui@nvidia-nim-free`, `direct.api@ollama`, `opencode.cli@openrouter`, `claude.cli@anthropic+corp-auth`) that every run path resolves through the same seam it resolves record ids today, so `{providerId, model, effort}` selections keep their shape.

## Context

- Every consumer resolves by id: `promptRunner.js:1637` (`effectiveProvider.type` → cli/api/tui runners), `agentProviderResolution.js` (task/role/active precedence), `aiToolkit/providerStatus.js:537 getFallbackProvider`, `tuiShellLaunch.js`, public review selection, `appTaskProviderPin.js`, `callerModePolicy.js` (`agent-harness`/`cli-harness`/`direct-api`/`any-text`).
- The toolkit is self-contained; host behavior is injected (the graph store seam in `providerGraph.js`, `withGatewayApiKey` in `aiToolkit/providers.js:54`). `getProvider(id)`/`getAllProviders()` are the lookup surface.
- Record ids match `^[a-z0-9][a-z0-9-]*$` (`aiToolkit/validation.js:75`); `.` and `@` cannot appear, so composites never collide and are never persisted as records.
- Harness binary detection: `providerRuntimeInstaller.js` (`PROVIDER_RUNTIMES`, 60s TTL, `peekProviderRuntimeStatuses` cache-only).
- Peer wire: `metadata.provider/model/effort` cross verbatim (`peerCosSync.js:365`); no schema entry for providers.

## Proposed approach

1. **Harness enablement.** `settings.harnesses: { [harnessId]: { enabled: boolean } }` (settings service; no new file/table). `harnessEnabled(id)` = explicit setting if present, else detected on PATH (`peekProviderRuntimeStatuses`), `direct` always true. `reconcileHarnessEnablement()` is idempotent and called from settings save and boot (no provider calls).
**Bootstrap apps** live beside it: `settings.credentialBootstraps: { [slug]: {label, command, args[], argsSeparator?, setupCommand?, harnessNames: {[harnessId]: string}} }` with `credentialBootstrapSchema` in `lib/validation.js` (same field limits as today's inline object, `aiToolkit/validation.js:205`); `GET/PUT /api/providers/bootstraps` CRUD, no spawn on save.
2. **Composite grammar.** `server/lib/providerRef.js` (pure, browser-safe): `parseProviderRef(id) → {kind:'preset', id} | {kind:'composite', harnessId, method, serviceSlug} | null`, `formatCompositeId(...)`, `COMPOSITE_ID_RE = /^([a-z0-9-]+)\.(cli|tui|api)@([a-z0-9][a-z0-9-]*)(?:\+([a-z0-9][a-z0-9-]*))?$/` (the `+` suffix is valid only with `cli`/`tui`; `parseProviderRef` returns `bootstrapSlug|null`). `providerRefSchema` in `lib/validation.js` (mirrored in `aiToolkit/validation.js` for `runSchema.provider`) accepts either grammar; swap it in at every selection validator that currently uses a bare provider-id string (cosValidation task metadata, orchestration profile roles, task templates, llmRoutePin consumers, aiAssignments, quota burn, creative director, persistent mind, decks, FableLoom, catalog ingest, POST, mood boards, music video, character augment, feature agents, loops, app `taskTypeOverrides`). `activeProvider` (`providerActiveSchema`, `aiToolkit/validation.js:280`), `providerSchema.fallbackProvider`, and `taskTypeOverrides` (`lib/validation.js:281`) keep the preset-only record-id regex; a materialized composite is never validated through `providerSchema`.
3. **Resolver.** Host-injected `resolveCompositeProvider(ref) → record | null` into the toolkit: loads the service instance by slug (enabled, credentials/subscription present), checks `harnessEnabled`, `isCompatible`, method ∈ `harness.modes`, then `materializeRoute` with the bootstrap app resolved from the suffix (rejects an unknown slug, a suffix on `api`, and a missing suffix on a service whose credential is `via: 'bootstrap'`) and defaults (no model pin unless the composite is paired with one; `defaultModel` = the service catalog's first entry only when the run passes no model — mirror today's behavior for records with no default). Cache per (composite, service revision, settings revision); invalidate on service/settings change. `getProvider(compositeId)` returns the materialized record with the credential attached **non-enumerably** (same discipline as `attachGatewaySiblingKey`) so it never serializes.
4. **Run paths.** Tests that execute a composite through each path with the real argv builders and a fake spawn/fetch: `pi.cli@nvidia-nim`, `pi.tui@nvidia-nim` (Shell launch too), `direct.api@nvidia-nim`, `opencode.cli@openrouter`, `claude.cli@ollama`, `codex.cli@lmstudio`, and a rejection for `claude.cli@nvidia-nim` (incompatible) and for a disabled harness; plus `claude.cli@anthropic+corp-auth` spawning ` run claude-code -- …` through `resolveCliSpawn` with process-group teardown, and the same composite under a public-review posture spawning `claude` unwrapped (existing skip). Fallback: `getFallbackProvider` may return a composite only when the caller's `allowedModes` admit its method (existing eligibility intersection, #6368).
5. **Catalog endpoint.** `GET /api/providers/catalog` → `{ harnesses: [{id, label, modes, enabled, detected, version}], services: [...sanitized instances...], bootstraps: [{slug, label, harnessNames}], compatibility: {[harnessId]: [serviceSlug...]}, effortLevels: {[harnessId]: string[]|null}, effortLevelsByModel: {[harnessId]: {[model]: string[]}}, presets: [...] }`. `GET /api/providers` stays presets-only so the 50 `useProviderModels` importers are unaffected until child 5. Status/readiness endpoints accept a composite id and answer from its parts.
6. `effortLevelsForProvider(provider, model)` reads `provider.harnessId` first (materialized records carry it), command sniffing second.

## Acceptance criteria

- [ ] With `pi` on PATH and an enabled `nvidia-nim` service holding a key, a CoS task pinned to `pi.tui@nvidia-nim` spawns `pi` with `--provider`/`--model` and the NVIDIA key in env, with no `pi-*nvidia*` record present; the same for `direct.api@nvidia-nim` via `executeApiRun`.
- [ ] Disabling the harness or the service makes the composite ineligible; existing saved selections naming it stay visible with a reason (never substituted); re-enabling restores them.
- [ ] `providerRefSchema` accepts `claude-code`, `pi.tui@nvidia-nim-free` and `claude.cli@anthropic+corp-auth`, rejects `pi.tui@Nvidia`, `pi.tui@`, `pi.gui@x`, `direct.api@ollama+corp-auth`; `PUT /api/providers/active`, a record `fallbackProvider`, and an app `taskTypeOverrides` pin each reject a composite with a 400 naming the preset-only rule.
- [ ] A composite never appears in `data/providers.json`, in any `GET` response as a credential-bearing record, or in a log line with its key.
- [ ] A peer receiving a task with `metadata.provider = 'pi.tui@nvidia-nim'` and no such service resolves to an unresolved pin with the existing reason path; no `schemaVersions.js` change.
- [ ] Boot performs no provider calls; the catalog endpoint is derived from cache and settings only.

## Out of scope

Structural preset fields and migration (child 4), any React change (child 5/6).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.