monad-developers / monad-developers/ultrafuzz

Support provider-agnostic coding harnesses via tight Smithers adapters

Aperta
#653 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

deferred-post-v0.1 enhancement
Lingua principale
TypeScript
Stelle
85
Fork
18
Merge medio
11h 10m
PR unite (30g)
194

Descrizione

Umbrella for provider-agnostic harness support, rewritten around the tight-adapter ruling.

Governing constraint

Ultrafuzz carries tight adapters over Smithers' built-in harness registry and never re-implements Smithers internals. Smithers ships 14 built-in CLI harnesses and already exposes every binding axis as constructor options: harness = the agent class, provider = baseURL + apiKey, model = an opaque provider catalogue ID, protocol = the class. smithers agents capabilities --format json publishes each harness's binary, emittedFlags, optionMappings and resume kind; smithers agents doctor and smithers agents test already validate reachability.

A tight adapter is one ~76-line file in packages/runtime/src/templates/smithers/agents/, modelled on claude.tsx, that reads only auth / api_key_env / config_dir from its own [agents.<Ref>Agent] table, maps them onto existing Smithers options, wraps child env in workflowControlChildEnvironment and credentials in workflowControlCredentialValue, and does nothing else.

What this umbrella no longer includes

  • No new configuration schema. No [providers.*], no [harnesses.*], no independent harness / provider fields, no ProviderBinding / HarnessCapabilities / QualifiedHarnessBinding types, no binding validator. models.*.agent is already an open agent-ref schema and [agents.*] is already a generic record, so a new harness needs zero config work. (#658 closed; PR #682 rejected.)
  • No ultrafuzz-side capability contract. Capability truth is smithers agents capabilities / doctor. Ultrafuzz consumes it; it does not define a second one.
  • No G1-G10 conformance suite. Qualification for a pairing is: one real provider smoke run on a pinned CLI, plus two unit assertions (credential absent from argv, credential present in child env) and one filesystem assertion (no harness state outside the run-scoped roots).

Sequencing (children are now independent)

  • #659 — Pi + OpenRouter tight adapter
  • #660 — correct the DeepSeek pairing terminology (docs/comments only)
  • #661 — DeepSeek Harness (dsh): not a tight adapter — no Smithers registry entry exists. Owner decision required; recommended resolution is upstream.
  • #662 — OpenCode + OpenRouter tight adapter with run-scoped state
  • #663 — Modal image: pin the new harness packages and extend the existing command prober
  • #664 — closed; residual doc wording folded into #660

Codex remains the shipped default until a qualified alternative beats it on real-provider evidence. That is a default-selection decision, not an architecture claim.

Adding a harness — the whole checklist

  1. Confirm the harness is in smithers agents capabilities; record packageExport, binary, optionMappings, resume. If it is absent, stop — this is not an adapter task.
  2. Copy claude.tsx, change five things: the Smithers class import, the [agents.<Ref>Agent] table name, the default auth mode, the default api_key_env, and the unattended-execution posture.
  3. Register in agents/index.tsx (agentFactories) and in AGENT_TEMPLATES in packages/runtime/src/init.ts.
  4. Map ref -> binary in AGENT_EXECUTABLES in packages/runtime/src/doctor.ts, using surface.binary.
  5. Add tests next to the existing ones: index export, agentFactories entry, no module-level construction, and a generated-adapter load test asserting the env passed through workflowControlChildEnvironment.
  6. Touch packages/config only for a hard constraint the harness genuinely rejects. Never add per-harness env allow-list entries to the runtime layer.

Review gate: under ~100 lines, no fs beyond readFileSync, no fetch, no argv indexing, no usage arithmetic, no session handling, no hardcoded model names or context sizes.

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Direzione di ricerca

Consideralo un’attività ombrello anziché una singola attività di implementazione: il lavoro indipendente è tracciato in #659, #660, #661, #662 e #663. Per uno specifico adapter, esamina prima smithers agents capabilities e packages/runtime/src/templates/smithers/agents/claude.tsx, quindi segui le posizioni elencate per la registrazione, il doctor mapping e i test. Il lavoro è completato quando l’elemento figlio selezionato supera il rigido gate di revisione dell’adapter e i controlli di qualificazione indicati.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
typescript
Ambito
cli, devtools, tooling
Tipo di issue
Funzionalità
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Tranquilla
Chiarezza
Abbastanza chiara
Idoneità per principianti
28/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.