OpenHands / OpenHands/software-agent-sdk

condenser: split summarization prompt into system + user messages

Open
#5,142 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

architecture enhancement llm memory
Dominant language
Python
Stars
1.1k
Forks
539
Avg merge
1d 19h
Merged PRs (30d)
137

Description

Summary

The LLMSummarizingCondenser sends its summarization request to the condenser's own LLM as a single user message that bundles both the steering instructions and the forgotten-event payload (see openhands/sdk/context/condenser/prompts/summarizing_prompt.j2).

On the OpenAI Responses API path, the event data lands in the wrong slot: a lone user message carrying the full instructions + payload is fine for Chat Completions, but it conflates "how to summarize" (steering) with "what to summarize" (data). More importantly, if the steering instructions were ever moved to system, a lone system message serializes to the instructions field with empty input (Message.to_responses_value returns a string for system; message_to_responses_dict returns [] for system), so the payload-to-summarize would never appear as a real input item.

Proposed change

Split the single summarizing_prompt.j2 template into two and send a canonical system + user message pair:

  • summarizing_system.j2 -> system message (the steering instructions: "You are maintaining a context-aware state summary...")
  • summarizing_events.j2 -> user message (the <EVENT> payload + "Now summarize the events using the rules above.")

This keeps the steering header in the provider's instructions/system slot and the event data in the input/user slot — the canonical shape for both Chat Completions and Responses. It also plays well with the subscription/Codex transport (transform_for_subscription), which prepends system chunks onto the first user message.

The change is isolated to the condenser's separate side-channel call (self.llm.generate(..., store=False)); it does not touch the agent's main conversation history.

Context

Discussion of how condensation works and what messages are sent to the condenser LLM. The condenser makes an independent, single-turn call to its own LLM with a freshly-built message list (no agent system prompt, no tools, no prior turns), so a system+user split is safe and semantically cleaner than the current single-user-message approach.

This issue was created by an AI agent (OpenHands) on behalf of @juanmichelini.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with openhands/sdk/context/condenser/prompts/summarizing_prompt.j2 and the LLMSummarizingCondenser call to self.llm.generate(..., store=False). Split the steering instructions and event payload into summarizing_system.j2 and summarizing_events.j2, then send them as a canonical system + user pair. Done means the steering text reaches the provider's instructions/system slot while the event payload remains in the input/user slot for both transport paths.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
73/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.