electric-sql / electric-sql/electric

agents-runtime: openai reasoning items dropped across wakes — LLMMessage schema gap

Open
#4,390 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
10.4k
Forks
375
Avg merge
3d 1h
Merged PRs (30d)
18

Description

## Summary

When a long-lived agents-runtime entity uses an OpenAI **reasoning** model (e.g. `gpt-5.5`, `o`-series) on a multi-wake conversation, the second wake fails with:

\`\`\`
404 Item with id 'rs_' not found.
Items are not persisted when \`store\` is set to false.
\`\`\`

The underlying LLM library (\`@mariozechner/pi-ai\`) handles reasoning correctly. The break is a schema gap in \`@electric-ax/agents-runtime\` that drops the encrypted reasoning blob between wakes — but keeps the paired \`function_call\` id, so OpenAI gets an unmatched \`fc_\` and rejects it.

## Root cause

1. pi-ai sends \`include: [\"reasoning.encrypted_content\"]\` for reasoning providers (\`pi-ai/dist/providers/openai-responses.js:178\`). The Responses API returns each \`rs_\` item with its full encrypted payload.

2. pi-ai stashes that blob as a JSON string in \`block.thinkingSignature\` (\`pi-ai/dist/providers/openai-responses-shared.js:367-369\`) and, on the next request, \`JSON.parse()\`s it back into the \`input\` array (\`openai-responses-shared.js:109-114\`). This is the contract for stateless reasoning — no \`store: true\` required.

3. **agents-runtime's \`LLMMessage\` schema has nowhere to persist \`thinkingSignature\`.** In \`packages/agents-runtime/src/types.ts:216-245\`:

- \`LLMAssistantMessage\` carries \`content: string\`. No \`thinkingSignature\`, no \`reasoning\` variant, no \`encrypted_content\` field.
- \`pi-adapter.ts:86-160\` \`toAgentHistory()\` rebuilds \`AgentMessage[]\` from \`LLMMessage[]\` on every wake. The assistant case (lines 107-122) only emits \`{type: 'text'}\` blocks. The pi-ai reasoning round-trip is broken at exactly the persistence boundary.

4. **But the paired \`function_call\` id (\`fc_\`) IS preserved** (\`pi-adapter.ts:124-144\`). OpenAI's Responses API tracks \`fc_\`↔\`rs_\` pairs and rejects an \`fc_\` submission without its companion \`rs_\`. So on the second wake, pi-adapter sends the \`function_call\` id without the reasoning item, and the API 404s on the missing \`rs_\`.

## Why the bundled paths don't hit this

The built-in catalog (\`packages/agents/src/model-catalog.ts\`) avoids the path by default:

- Anthropic is preferred when \`ANTHROPIC_API_KEY\` is set (\`model-catalog.ts:209-222\`). Anthropic returns no \`rs_\` items.
- The OpenAI fallback is \`DEFAULT_OPENAI_MODEL = 'gpt-4.1'\` (\`model-catalog.ts:49\`), which has \`reasoning: false\` in the generated catalog → \`withProviderPayloadDefaults\` short-circuits at \`model-catalog.ts:148-152\` and no \`reasoning.effort\` is sent → no \`rs_\` items.
- \`openai-codex\` paths run single-turn or rely on the codex token's own store semantics.

The gap surfaces when:
- a host application explicitly picks an OpenAI reasoning model (e.g. \`gpt-5.5\`) for an entity that
- runs a multi-wake conversation with tool calls (so a paired \`fc_\` lands in history).

## Repro

1. Register an entity-type with a model catalog whose default choice is \`{ provider: 'openai', id: 'gpt-5.5', reasoning: true }\`.
2. Spawn one instance. Have it issue a tool call on its first wake (so the wake history records an \`fc_\` paired with an \`rs_\`).
3. Send a follow-up message. On the second wake, \`pi-adapter.toAgentHistory\` rebuilds messages without the \`thinking\` block; the model call goes out carrying \`fc_\` but no \`rs_\`. Responses API returns \`404 Item with id 'rs_' not found\`.

## Proposed fix

Three coordinated changes in \`packages/agents-runtime/src/\`:

1. **\`types.ts:230\`** — add an optional \`thinkingSignature?: string\` to \`LLMAssistantMessage\`, or model a separate \`thinking\` content-variant alongside \`text\` and \`tool_use\`. Either shape will round-trip pi-ai's blob.
2. **\`outbound-bridge.ts\`** — capture pi-agent-core's \`thinking_end\` events (which carry \`contentSignature\`) and write them into the durable event stream alongside the assistant text.
3. **\`pi-adapter.ts:107-122\`** — in \`toAgentHistory\`, when an assistant message carries a \`thinkingSignature\`, prepend a \`{type: 'thinking', thinking: '', thinkingSignature}\` block before the text block so pi-ai sees it on rebuild.

After that, \`gpt-5.5\` (and other reasoning OpenAI models) should run multi-wake without errors, matching what the existing fc_/tool-result round-trip already does.

## Workaround for now

Set \`reasoning: false\` on the OpenAI catalog choice. The runtime then never asks for reasoning content, no \`rs_\` items are emitted, and the gap is invisible. This matches what the built-in \`gpt-4.1\` default already does — it's the only validated openai path today.

## Versions

- \`@electric-ax/agents-runtime\` 0.3.0
- \`@electric-ax/agents\` 0.4.4
- \`@mariozechner/pi-ai\` 0.70.2 — 0.70.6 (relevant dist files are byte-identical between these versions)
- \`@mariozechner/pi-agent-core\` 0.70.2 — 0.70.6

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.