OpenHands / OpenHands/software-agent-sdk
[Bug]: `Message.to_chat_dict` emits `cache_control` when caching is disabled
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 539
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 137
Description
Is there an existing issue for the same bug?
- I have searched existing issues and this is not a duplicate.
Bug Description
In openhands-sdk/openhands/sdk/llm/message.py, Message.to_chat_dict can emit a cache_control marker even when called with cache_enabled=False.
When list serialization is selected for another reason, such as function_calling_enabled=True, _list_serializer processes existing cache_prompt=True content without receiving or checking cache_enabled.
For a tool message, the marker is promoted to the top-level message dictionary despite caching being disabled.
Expected Behavior
Given a tool message containing TextContent(cache_prompt=True), calling to_chat_dict with cache_enabled=False and function_calling_enabled=True should return the serialized tool message without any cache marker:
{
"content": [{"type": "text", "text": "Tool response"}],
"role": "tool",
"tool_call_id": "call_123",
"name": "test_tool",
}
cache_enabled=False should suppress both message-level and content-level cache_control fields, even when another feature requires list serialization.
Actual Behavior
Message.to_chat_dict returns:
{
"content": [{"type": "text", "text": "Tool response"}],
"role": "tool",
"cache_control": {"type": "ephemeral"},
"tool_call_id": "call_123",
"name": "test_tool",
}
The returned message contains a top-level cache marker even though cache_enabled is False.
Steps to Reproduce
-
Check out Software Agent SDK
mainat commit007721b3d2bfccd1469f8008a514af13b7ae1b71. -
Add the following test to
tests/sdk/llm/test_message.py:
def test_message_tool_role_omits_cache_prompt_when_cache_disabled():
from openhands.sdk.llm.message import Message, TextContent
message = Message(
role="tool",
content=[TextContent(text="Tool response", cache_prompt=True)],
tool_call_id="call_123",
name="test_tool",
)
result = message.to_chat_dict(
**{
**DEFAULT_SERIALIZATION_OPTS,
"cache_enabled": False,
"function_calling_enabled": True,
}
)
assert "cache_control" not in result
assert "cache_control" not in result["content"][0]
- Run:
uv run pytest tests/sdk/llm/test_message.py -q
- Observe that the newly added assertion fails while the 30 existing tests pass.
Acceptance Criteria
-
cache_enabled=Falseprevents list serialization from emitting message-level or content-levelcache_controlfields. - Disabling caching remains effective when list serialization is selected by function calling or vision support.
- Existing
cache_enabled=Truebehavior for tool and non-tool messages remains unchanged. - Regression coverage includes a tool message with
cache_prompt=True,cache_enabled=False, andfunction_calling_enabled=True.
Installation Method
Source checkout; focused pytest reproduction after the documented development setup
If you selected "Other", please specify
No response
SDK Version
main@007721b3d2bfccd1469f8008a514af13b7ae1b71 (openhands-sdk 1.42.1)
Version Confirmation
- I have confirmed this bug exists on the LATEST version of OpenHands SDK
Python Version
3.13.2
Model Name (if applicable)
Not applicable; reproduced by directly invoking deterministic message-serialization code
Operating System
MacOS
Logs and Error Messages
tests/sdk/llm/test_message.py .....F......................... [100%]
FAILED tests/sdk/llm/test_message.py::test_message_tool_role_omits_cache_prompt_when_cache_disabled
AssertionError: assert 'cache_control' not in {'cache_control': {'type': 'ephemeral'}, 'content':
[{'text': 'Tool response', 'type': 'text'}], 'name': 'test_tool', 'role': 'tool', ...}
1 failed, 30 passed in 0.14s
Minimal Code Sample
from openhands.sdk.llm.message import Message, TextContent
message = Message(
role="tool",
content=[TextContent(text="Tool response", cache_prompt=True)],
tool_call_id="call_123",
name="test_tool",
)
result = message.to_chat_dict(
cache_enabled=False,
vision_enabled=False,
function_calling_enabled=True,
force_string_serializer=False,
send_reasoning_content=False,
)
print(result)
# {
# 'content': [{'type': 'text', 'text': 'Tool response'}],
# 'role': 'tool',
# 'cache_control': {'type': 'ephemeral'},
# 'tool_call_id': 'call_123',
# 'name': 'test_tool',
# }
Screenshots and Additional Context
At the tested commit, to_chat_dict selects list serialization when any of caching, vision, or function calling is enabled:
if not force_string_serializer and (
cache_enabled or vision_enabled or function_calling_enabled
):
message_dict = self._list_serializer(vision_enabled=vision_enabled)
However, _list_serializer does not receive cache_enabled. It calls item.to_llm_dict(), which emits cache_control whenever the content object's cache_prompt field is true:
item_dicts = item.to_llm_dict()
For tool messages, it then unconditionally records that caching is requested and promotes the marker to the message level:
if self.role == "tool" and item.cache_prompt:
role_tool_with_prompt_caching = True
for d in item_dicts:
d.pop("cache_control", None)
The public LLM formatting path passes the current cache setting into to_chat_dict. APIBasedCritic also calls this method with cache_enabled=False and function_calling_enabled=True. Consequently, a message already carrying cache_prompt=True can emit a cache marker through these serialization paths even though caching is explicitly disabled.
One possible fix direction is to pass cache_enabled into _list_serializer and use it to gate both content-level cache markers and the tool-message promotion logic. Existing behavior when cache_enabled=True should remain unchanged.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in openhands-sdk/openhands/sdk/llm/message.py by tracing Message.to_chat_dict into _list_serializer, then run the focused reproduction in tests/sdk/llm/test_message.py with pytest. Done means the tool-message case omits both cache_control fields when caching is disabled, while existing enabled-cache behavior and the surrounding tests remain passing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design, testing-qa
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 86/100