Hermes sandboxes cannot use Anthropic-native (anthropic-messages) custom endpoints — with working patch

# Hermes sandboxes cannot use Anthropic-native (anthropic-messages) custom endpoints — unlock request

## Summary
NemoClaw already contains the complete code path for routing Hermes to a custom Anthropic-compatible endpoint natively (`anthropic-messages`), but a set of guards force the "managed openai-completions frontend" instead. On endpoints that only serve the Anthropic Messages API (e.g. corporate AI gateways exposing Claude-style surfaces at `/anthropic/v1/messages` but no OpenAI chat-completions surface), this makes otherwise-supported models unusable for Hermes sandboxes.

## Evidence that the native path is already built
- `src/lib/hermes-managed-route.ts`: `hermesApiMode()` maps `anthropic-messages` → `anthropic_messages`, and `applyHermesManagedRoute()` writes `api_mode` into both `model:` and `custom_providers:` blocks of the Hermes config.
- `src/lib/inference/config.ts`: the provider switch for `compatible-anthropic-endpoint` already has the native branch (`providerKey = "anthropic"`, `inferenceBaseUrl = "https://inference.local"`, `inferenceApi = "anthropic-messages"`) — but it is unreachable because `resolveAgentInferenceApi()` hardcodes `"openai-completions"` for hermes+compatible-anthropic-endpoint (line ~187).
- Hermes itself (tested v0.20.6) natively supports `api_mode: anthropic_messages`, ships the `anthropic` SDK, and honors `ANTHROPIC_BASE_URL` (`hermes_cli/auth.py`).

## The interlock chain (what blocks it today)
1. `src/lib/actions/inference-set.ts` (~L885): hard throws — "Hermes custom Anthropic endpoints require the managed openai-completions frontend."
2. `src/lib/inference/config.ts` `resolveAgentInferenceApi()` (~L187): silently downgrades a requested `anthropic-messages` to `openai-completions` for hermes+compatible-anthropic-endpoint, so even a patched guard writes the wrong `preferredInferenceApi` into the registry.
3. With both patched, `inference-set-provider.ts` `assertProviderOwnership()` refuses to replace a provider whose live gateway binding was created with the OpenAI family ("malformed, foreign, or does not match durable custom-endpoint provenance"), and `openshell provider delete` refuses because the sandbox is attached. The only reconciliation offered is re-onboard.
4. Re-onboarding with the anthropic-compatible provider then dies in the OpenShell CLI's own route verification, which probes `{route}/v1/chat/completions` from the host (that URL is sandbox-internal, and the surface wouldn't speak chat-completions anyway). Other provider paths (bedrock, openrouter, local) pass `--no-verify` to `openshell inference set`; this one does not.

## Why it matters
Cost and coverage: some gateways expose certain models only via Anthropic Messages (in our case Kimi K3 at roughly 5-10x lower cost than the GPT models available on the OpenAI surface). OpenClaw sandboxes can use these (the validation guidance itself says "switch to an Anthropic-native agent: --agent openclaw"); Hermes users cannot, despite Hermes supporting the protocol.

## Suggested fix
1. Let `resolveAgentInferenceApi` honor an explicit `anthropic-messages` request for hermes (keep openai-completions as default).
2. Drop or downgrade the inference-set guard to a warning.
3. Add a provider recreate/migrate flow (or allow `provider delete` + re-create with sandbox detach) so the family can change without full re-onboard.
4. Pass `--no-verify` (or probe `/v1/messages`) on the anthropic-compatible onboard path.
5. Emit `api_mode: anthropic_messages` via the existing `applyHermesManagedRoute` (already works once 1+2 are fixed).

## Environment
- NemoClaw: main @ 70cfff5f9 (v0.0.123-13)
- OpenShell: 0.0.116 (docker driver, macOS arm64)
- Hermes Agent: v0.20.6
- Endpoint: corporate AI gateway serving Anthropic Messages at `/anthropic/v1/messages` (verified working with curl; 200 + real completion)

## Notes
- Local patches to (1) and (2) allowed the route to be set and synced; remaining interlocks (3)/(4) still block end-to-end.
- Related papercut found while testing: NemoClaw's dashboard/forward launch under XDG_CONFIG_HOME isolation fails its own `owns()` verification (child binds fine when spawned manually with identical args; only fails when spawned by onboard). May be an env-filtering nuance in `buildSubprocessEnv` (XDG_STATE_HOME/XDG_DATA_HOME not forwarded).

---

## UPDATE 2026-09-12: SOLVED LOCALLY — full patch set + proof

End-to-end proof: `POST /v1/chat/completions` to the Hermes managed API returned a real completion from `anthropic.kimi-k3`; supervisor log confirms `Success anthropic.kimi-k3 via https:///anthropic [POST /v1/messages]` — native Anthropic Messages over the OpenShell L7 proxy, no protocol translation.

### Final patch set (5 patches; all also applied to dist/ builds)

1. **`src/lib/actions/inference-set.ts`** — remove the hard throw "Hermes custom Anthropic endpoints require the managed openai-completions frontend" (guard 1).
2. **`src/lib/inference/config.ts`** — `resolveAgentInferenceApi`: honor an explicit `anthropic-messages` request for hermes+compatible-anthropic-endpoint instead of forcing openai-completions (guard 2).
3. **`src/lib/actions/inference-set-provider.ts`** — `assertProviderOwnership`: accept the dual-surface transitional binding (openai-typed provider with COMPATIBLE_ANTHROPIC_API_KEY + both base-url keys) so the family switch can proceed (guard 3).
4. **`src/lib/onboard/inference-providers/remote.ts`** — pass `--no-verify` for `compatible-anthropic-endpoint` route application; the host-side probe cannot reach the sandbox-internal bridge URL (guard 4a). Same file: accept the native anthropic binding (type "anthropic" + ANTHROPIC_BASE_URL) in the hermes provider-surface check (guard 4b, in inference-set.ts ~L780).
5. **`src/lib/inference/probe-anthropic.ts`** — raise the tool-use probe budget from `max_tokens: 64` to 1024: thinking models (kimi-k3, reasoning models generally) spend tokens on `thinking` blocks before emitting `tool_use`, so a 64-token budget false-negatives on exactly the models this path serves (guard 5).

### Operational note for the recreate flow
Switching an existing sandbox's provider family also requires recreating the gateway provider record with the correct type (`openai` → `anthropic`): `openshell sandbox provider detach` → `provider delete` → `provider create --type anthropic --credential COMPATIBLE_ANTHROPIC_API_KEY --config ANTHROPIC_BASE_URL=` → `sandbox provider attach`. A `nemoclaw inference set --migrate-provider` flow (or onboard handling it) would remove this manual step.

### Validation probe note
The strict SSE validator is good; it just needs the bigger token budget for thinking models (patch 5). The gateway's stream was textbook (message_start → content blocks → message_delta(stop_reason) → message_stop).

## Proposed patch

kimi-anthropic.patch (click to expand)

```diff
diff --git a/agents/hermes/config/hermes-env.ts b/agents/hermes/config/hermes-env.ts
index b9bef016f..81b6517a3 100644
--- a/agents/hermes/config/hermes-env.ts
+++ b/agents/hermes/config/hermes-env.ts
@@ -11,7 +11,15 @@ export function buildHermesEnvLines(
settings: HermesBuildSettings,
env: NodeJS.ProcessEnv = process.env,
): string[] {
- const envLines = ["API_SERVER_PORT=18642", "API_SERVER_HOST=127.0.0.1"];
+ const envLines = [
+ "API_SERVER_PORT=18642",
+ "API_SERVER_HOST=127.0.0.1",
+ // Seal the agent venv: without PyPI egress, Hermes' lazy-deps path hangs
+ // the TUI at startup (uv pip install edge-tts blocks forever) — observed
+ // 2026-09-10 on zende-hermes. The published Hermes Docker image sets this;
+ // NemoClaw sandboxes must too.
+ "HERMES_DISABLE_LAZY_INSTALLS=1",
+ ];

for (const { envKey, placeholder } of settings.messagingCredentialPlaceholders) {
envLines.push(`${envKey}=${placeholder}`);
diff --git a/src/lib/actions/inference-set-provider.ts b/src/lib/actions/inference-set-provider.ts
index 361abc59e..21370e40f 100644
--- a/src/lib/actions/inference-set-provider.ts
+++ b/src/lib/actions/inference-set-provider.ts
@@ -194,7 +194,21 @@ function assertProviderOwnership(options: {
1,
);
}
+ // PATCH(jlanders 2026-09-11): kimi-k3 unlock — accept the dual-surface
+ // transitional binding for compatible-anthropic-endpoint: an openai-typed
+ // provider carrying COMPATIBLE_ANTHROPIC_API_KEY plus both base-url config
+ // keys is the state an anthropic onboard leaves behind; it is owned by this
+ // toolchain, not foreign.
+ const dualSurfaceAnthropicTransition =
+ observation.metadata !== null &&
+ observation.metadata.name === providerName &&
+ providerName === "compatible-anthropic-endpoint" &&
+ observation.metadata.type === "openai" &&
+ observation.metadata.credentialKeys.includes("COMPATIBLE_ANTHROPIC_API_KEY") &&
+ observation.metadata.configKeys.includes("ANTHROPIC_BASE_URL") &&
+ observation.metadata.configKeys.includes("OPENAI_BASE_URL");
if (
+ !dualSurfaceAnthropicTransition &&
!matchesGatewayProviderBinding(
observation.metadata,
expectedShape(providerName, surface, credentialEnv),
diff --git a/src/lib/actions/inference-set.ts b/src/lib/actions/inference-set.ts
index ac65a15aa..87359a14f 100644
--- a/src/lib/actions/inference-set.ts
+++ b/src/lib/actions/inference-set.ts
@@ -778,14 +778,25 @@ async function assertHermesCompatibleAnthropicOpenAiProvider(
target: { kind: "named", gatewayName },
providerName: provider,
});
+ // PATCH(jlanders 2026-09-11): accept the dual-surface binding too — after an
+ // anthropic-messages onboard the provider carries both OPENAI_BASE_URL and
+ // ANTHROPIC_BASE_URL config keys; requiring exactly one blocks the native
+ // anthropic route. Name+type+credential still must match.
if (
result.ok &&
- matchesGatewayProviderBinding(result.value, {
+ (matchesGatewayProviderBinding(result.value, {
name: provider,
type: "openai",
credentialKey: "COMPATIBLE_ANTHROPIC_API_KEY",
configKey: "OPENAI_BASE_URL",
- })
+ }) ||
+ // PATCH(jlanders 2026-09-11): kimi-k3 unlock — also accept the native
+ // anthropic binding (type "anthropic" + ANTHROPIC_BASE_URL), which is the
+ // correct shape for anthropic-messages routing.
+ (result.value.name === provider &&
+ result.value.type === "anthropic" &&
+ result.value.credentialKeys.includes("COMPATIBLE_ANTHROPIC_API_KEY") &&
+ result.value.configKeys.includes("ANTHROPIC_BASE_URL")))
) {
return;
}
@@ -875,18 +886,12 @@ async function runInferenceSetWithoutHostLock(
const explicitOrRecordedInferenceApi =
explicitInferenceApi ??
(entry.provider === provider ? (entry.preferredInferenceApi ?? null) : null);
- if (
- agentName === "hermes" &&
- provider === "compatible-anthropic-endpoint" &&
- explicitInferenceApi !== null &&
- explicitInferenceApi !== "openai-completions"
- ) {
- throw new InferenceSetError(
- "Hermes custom Anthropic endpoints require the managed openai-completions frontend. " +
- "Set --inference-api openai-completions or omit --inference-api so NemoClaw selects it.",
- 2,
- );
- }
+ // PATCH(jlanders 2026-09-11): allow hermes + compatible-anthropic-endpoint +
+ // anthropic-messages so zende-hermes can use the ai-gateway /anthropic
+ // surface (kimi-k3) natively. Hermes v0.20.6 supports Anthropic providers
+ // with ANTHROPIC_BASE_URL; OpenShell 0.0.116 proxies anthropic-messages
+ // (proven by prime-claw). Revert by restoring inference-set.ts.bak-anthropic-allow.
+ if (false) {
const hasExplicitCustomRoute = Boolean(
options.endpointUrl || options.credentialEnv || options.inferenceApi,
);
diff --git a/src/lib/inference/config.ts b/src/lib/inference/config.ts
index 32d86014f..45415b99a 100644
--- a/src/lib/inference/config.ts
+++ b/src/lib/inference/config.ts
@@ -184,8 +184,10 @@ export function resolveAgentInferenceApi(
provider: string | null | undefined,
preferredInferenceApi: string | null,
): string | null {
+ // PATCH(jlanders 2026-09-11): honor anthropic-messages for hermes +
+ // compatible-anthropic-endpoint (kimi-k3 unlock; see inference-set.ts patch).
return agentName === "hermes" && provider === "compatible-anthropic-endpoint"
- ? "openai-completions"
+ ? (preferredInferenceApi ?? "openai-completions")
: preferredInferenceApi;
}

diff --git a/src/lib/inference/probe-anthropic.ts b/src/lib/inference/probe-anthropic.ts
index 3b8822f78..5b210fd8e 100644
--- a/src/lib/inference/probe-anthropic.ts
+++ b/src/lib/inference/probe-anthropic.ts
@@ -97,7 +97,7 @@ function anthropicMessagesPayload(
): string {
return JSON.stringify({
model,
- max_tokens: requireToolCalling ? 64 : 16,
+ max_tokens: requireToolCalling ? 1024 : 16, // PATCH(jlanders): thinking models (e.g. kimi-k3) spend tokens on reasoning before tool_use
...(stream ? { stream: true } : {}),
messages: [
{
diff --git a/src/lib/onboard/inference-providers/remote.ts b/src/lib/onboard/inference-providers/remote.ts
index ce600dbed..21ee27afb 100644
--- a/src/lib/onboard/inference-providers/remote.ts
+++ b/src/lib/onboard/inference-providers/remote.ts
@@ -385,7 +385,11 @@ export async function setupRemoteProviderInference(
return exitProcess(providerResult.status || 1);
}
const argsv = ["inference", "set"];
- if (config.skipVerify || gatewayEndpointUrl !== resolvedEndpointUrl) {
+ // PATCH(jlanders 2026-09-11): the Anthropic-compatible bridge URL is
+ // sandbox-internal and speaks Messages API, not chat/completions — the
+ // host-side openshell verify probe can never succeed on it. Skip it
+ // (mirrors bedrock/openrouter paths). Part of the kimi-k3 unlock.
+ if (config.skipVerify || gatewayEndpointUrl !== resolvedEndpointUrl || provider === "compatible-anthropic-endpoint") {
// Host-side verification cannot resolve the sandbox-only bridge URL.
argsv.push("--no-verify");
}

```

Start with src/lib/actions/inference-set.ts and src/lib/inference/config.ts to trace the Hermes provider guard and inference API selection, then follow provider ownership in src/lib/actions/inference-set-provider.ts and onboarding in src/lib/onboard/inference-providers/remote.ts. Review src/lib/inference/probe-anthropic.ts and the proposed patch, then reproduce the native endpoint flow. Done means Hermes uses anthropic-messages end to end, including provider reconciliation and route validation.

Written by the indexing model from the issue text.