monad-developers / monad-developers/ultrafuzz
Support provider-agnostic coding harnesses via tight Smithers adapters
Dieses Issue hat noch niemand übernommen.
- Vorherrschende Sprache
- TypeScript
- Sterne
- 85
- Forks
- 18
- Ø Merge
- 11 Std. 10 Min.
- Gemergte PRs (30 T.)
- 194
Beschreibung
Umbrella for provider-agnostic harness support, rewritten around the tight-adapter ruling.
Governing constraint
Ultrafuzz carries tight adapters over Smithers' built-in harness registry and never re-implements Smithers internals. Smithers ships 14 built-in CLI harnesses and already exposes every binding axis as constructor options: harness = the agent class, provider = baseURL + apiKey, model = an opaque provider catalogue ID, protocol = the class. smithers agents capabilities --format json publishes each harness's binary, emittedFlags, optionMappings and resume kind; smithers agents doctor and smithers agents test already validate reachability.
A tight adapter is one ~76-line file in packages/runtime/src/templates/smithers/agents/, modelled on claude.tsx, that reads only auth / api_key_env / config_dir from its own [agents.<Ref>Agent] table, maps them onto existing Smithers options, wraps child env in workflowControlChildEnvironment and credentials in workflowControlCredentialValue, and does nothing else.
What this umbrella no longer includes
- No new configuration schema. No
[providers.*], no[harnesses.*], no independentharness/providerfields, noProviderBinding/HarnessCapabilities/QualifiedHarnessBindingtypes, no binding validator.models.*.agentis already an open agent-ref schema and[agents.*]is already a generic record, so a new harness needs zero config work. (#658 closed; PR #682 rejected.) - No ultrafuzz-side capability contract. Capability truth is
smithers agents capabilities/doctor. Ultrafuzz consumes it; it does not define a second one. - No G1-G10 conformance suite. Qualification for a pairing is: one real provider smoke run on a pinned CLI, plus two unit assertions (credential absent from argv, credential present in child env) and one filesystem assertion (no harness state outside the run-scoped roots).
Sequencing (children are now independent)
- #659 — Pi + OpenRouter tight adapter
- #660 — correct the DeepSeek pairing terminology (docs/comments only)
- #661 — DeepSeek Harness (
dsh): not a tight adapter — no Smithers registry entry exists. Owner decision required; recommended resolution is upstream. - #662 — OpenCode + OpenRouter tight adapter with run-scoped state
- #663 — Modal image: pin the new harness packages and extend the existing command prober
- #664 — closed; residual doc wording folded into #660
Codex remains the shipped default until a qualified alternative beats it on real-provider evidence. That is a default-selection decision, not an architecture claim.
Adding a harness — the whole checklist
- Confirm the harness is in
smithers agents capabilities; recordpackageExport,binary,optionMappings,resume. If it is absent, stop — this is not an adapter task. - Copy
claude.tsx, change five things: the Smithers class import, the[agents.<Ref>Agent]table name, the default auth mode, the defaultapi_key_env, and the unattended-execution posture. - Register in
agents/index.tsx(agentFactories) and inAGENT_TEMPLATESinpackages/runtime/src/init.ts. - Map ref -> binary in
AGENT_EXECUTABLESinpackages/runtime/src/doctor.ts, usingsurface.binary. - Add tests next to the existing ones: index export,
agentFactoriesentry, no module-level construction, and a generated-adapter load test asserting the env passed throughworkflowControlChildEnvironment. - Touch
packages/configonly for a hard constraint the harness genuinely rejects. Never add per-harness env allow-list entries to the runtime layer.
Review gate: under ~100 lines, no fs beyond readFileSync, no fetch, no argv indexing, no usage arithmetic, no session handling, no hardcoded model names or context sizes.
Beitragsleitfaden
Erste Schritte
- Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
- Forke das Repository und arbeite in einem Branch.
- Öffne einen Pull Request, der die Issue-Nummer nennt.
Rechercherichtung
Betrachte dies als einen übergeordneten Rahmen und nicht als eine einzelne Implementierungsaufgabe: Die unabhängige Arbeit wird in #659, #660, #661, #662 und #663 verfolgt. Untersuche für einen bestimmten Adapter zunächst smithers agents capabilities und packages/runtime/src/templates/smithers/agents/claude.tsx und folge anschließend den aufgeführten Stellen für Registrierung, doctor mapping und Tests. Als erledigt gilt die Aufgabe, wenn das ausgewählte Kind die strengen Adapter-Review-Gates und die angegebenen Qualifikationsprüfungen erfüllt.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- typescript
- Bereich
- cli, devtools, tooling
- Issue-Typ
- Feature
- Schwierigkeit
- 5/5
- Geschätzter Aufwand
- Über eine Woche
- Aktivitätsstatus
- Ruhig
- Klarheit
- Größtenteils klar
- Anfängerfreundlichkeit
- 28/100