langchain-ai / langchain-ai/deepagentsjs
Bug: SummarizationMiddleware internal model call tokens leak into `agent.stream()` output
- Dominant language
- TypeScript
- Stars
- 1.6k
- Forks
- 272
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 38
Description
## Bug: SummarizationMiddleware internal model call tokens leak into `agent.stream()` output
### Description
When the summarization middleware triggers, the tokens from its internal `chatModel.invoke()` call **leak into the parent agent's streaming output**. Users see summary generation text (e.g. "Here is a concise summary of the conversation...") mixed into the normal AI response stream. This affects **all stream modes** — `streamMode: 'messages'`, `streamMode: 'updates'`, and `streamEvents` alike. There is no way to distinguish summary output from normal AI output at the stream level — both appear as `msg.type === 'ai'` chunks with the same namespace and no distinguishing metadata.
(Note: `streamEvents` has the same issue — summary model tokens leak into the event stream as well.)
### Root Cause
`createSummarizationMiddleware` reuses the **same model instance** as the parent agent for its internal model call:
```js
// langsmith-wdF8zG42.js line 5773 / 5821
createSummarizationMiddleware({ backend })
// ⚠️ No `model` option passed → falls back to `request.model` (the agent's model instance)
```
This model instance already has **LangGraph streaming callbacks** attached (set up by `agent.stream()`). When `performSummarization` → `summarizeMessages` → `createSummary` calls `chatModel.invoke()`:
```js
// Inside createSummary()
const response = await chatModel.invoke([new HumanMessage({ content: prompt })]);
```
The callbacks inherited from the parent graph context capture every token and emit them through `agent.stream()` as `streamMode: 'messages'` chunks.
This is the same root cause as **LangChain Python PR #34501** (already merged): `ensure_config()` inherits callbacks from `var_child_runnable_config`.
### Relevant Code Path
```
agent.stream(input, { streamMode: ['messages', 'updates'], subgraphs: true })
→ wrap_model_call middleware
→ summarizationMiddleware.wrapModelCall()
→ performSummarization()
→ summarizeMessages()
→ createSummary()
→ chatModel.invoke([summarization prompt]) // ← TOKENS LEAK HERE
→ buildSummaryMessage(summaryText)
→ handler({ messages: [summary_HumanMessage, ...preserved] })
→ agent model generates actual response // ← NORMAL OUTPUT
→ Command({ update: { _summarizationEvent: {...} } })
```
### Observed Behavior
1. **Prompt logs confirm an extra model call**: A `TokenLogger` (`BaseCallbackHandler` attached to the model instance) captures the summarization prompt as a separate invocation
2. **Stream output contains summary text**: The summary AI message appears in `agent.stream()` as `msg.type === 'ai'` chunks
3. **No distinguishing metadata**: Both summary and normal AI chunks have `msg.name === undefined`, identical namespace, and no differentiating `additional_kwargs`. The second element of `data` (LangGraph metadata) is **completely identical** between summary and normal messages — no field can be used to tell them apart.
The user-side stream handling code below shows the problem: summary-leaked AI messages and normal replies go through the same branch with no way to distinguish them:
```ts
for await (const [namespace, mode, data] of stream) {
if (mode === 'messages') {
const msg = Array.isArray(data) ? data[0] : data
// data[1] is LangGraph metadata — identical for summary and normal messages
if (msg.type === 'ai' && !!msg.content) {
// Sub-agent output
if (namespace && namespace.length > 1) {
callbacks.onChunk({ content: msg.content, type: 'think' })
} else {
// ← Both leaked summary and normal reply end up here
// msg.name === undefined (same for both)
// msg.additional_kwargs — no difference
// namespace — same (main agent)
// data[1] metadata — completely identical
callbacks.onChunk({ content: msg.content, type: 'result' })
}
}
}
}
```
4. **After first summarization, it recurs**: `_summarizationEvent` is persisted to checkpoint, causing `getEffectiveMessages()` to prepend the summary `HumanMessage` on every subsequent call. This increases token count, re-triggering summarization repeatedly
5. **Summary HumanMessage IS properly tagged**: `buildSummaryMessage` correctly sets `additional_kwargs: { lc_source: "summarization" }` on the summary HumanMessage, but this only marks the **injected context message**, not the leaked AI output
### Stream data during summarization
```
[messages] AI chunk "Here is a concise summary..." ← LEAKED (no lc_source tag)
[messages] AI chunk "**Main Topics:**..." ← LEAKED
[messages] HumanMessage { lc_source: "summarization" } ← delimiter (properly tagged)
[messages] AI chunk "好呀,想聊什么?" ← normal reply
[messages] AI chunk "..." ← normal reply
[updates] { _summarizationEvent: { cutoffIndex: 14 } }
```
### Related
- **LangChain Python PR #34501** — Same bug, same root cause. Fixed by passing `config={"callbacks": []}` to all internal model calls in `SummarizationMiddleware`, `LLMToolSelectorMiddleware`, `ToolEmulatorMiddleware`. **Already merged.**
- **PR #34763** — Added `metadata: { lc_source: "summarization" }` for UI detection (Python side)
### Proposed Fix
Option A (align with Python fix): Pass `config: { callbacks: [] }` to the internal `chatModel.invoke()` call inside `createSummary()`:
```js
const response = await chatModel.invoke(
[new HumanMessage({ content: prompt })],
{ callbacks: [] } // ← prevent callback inheritance
);
```
Option B (alternative): Allow passing a separate `model` to `createSummarizationMiddleware()`, so it uses a different model instance without the parent graph's callbacks. Currently `createSummarizationMiddleware({ backend })` is called without `model`, causing fallback to `request.model`.
### Environment
- deepagents: `1.10.5`
- `@langchain/langgraph`: `1.4.6`
- `@langchain/core`: `1.2.1`
- LangGraph: with `SqliteSaver` checkpointer
- Model: DeepSeek (ChatDeepSeek), `maxInputTokens` lowered to `10000` via profile override
- Stream config: `{ streamMode: ['messages', 'updates'], subgraphs: true }`
Contributor guide
Research direction
Start at createSummarizationMiddleware and follow performSummarization through summarizeMessages to createSummary, where the internal chatModel.invoke() call is described. Reproduce with agent.stream using messages and updates modes, then verify that summarization text no longer appears as AI chunks while the normal response and summarization update remain.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100