OpenHands / OpenHands/software-agent-sdk

[Bug]: `Message.to_chat_dict` emits `cache_control` when caching is disabled

Open Beginner friendly
#4,511 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug llm priority:medium ready-for-dev
Dominant language
Python
Stars
1.1k
Forks
539
Avg merge
1d 19h
Merged PRs (30d)
137

Description

Is there an existing issue for the same bug?
  • I have searched existing issues and this is not a duplicate.
Bug Description

In openhands-sdk/openhands/sdk/llm/message.py, Message.to_chat_dict can emit a cache_control marker even when called with cache_enabled=False.

When list serialization is selected for another reason, such as function_calling_enabled=True, _list_serializer processes existing cache_prompt=True content without receiving or checking cache_enabled.

For a tool message, the marker is promoted to the top-level message dictionary despite caching being disabled.

Expected Behavior

Given a tool message containing TextContent(cache_prompt=True), calling to_chat_dict with cache_enabled=False and function_calling_enabled=True should return the serialized tool message without any cache marker:

{
    "content": [{"type": "text", "text": "Tool response"}],
    "role": "tool",
    "tool_call_id": "call_123",
    "name": "test_tool",
}

cache_enabled=False should suppress both message-level and content-level cache_control fields, even when another feature requires list serialization.

Actual Behavior

Message.to_chat_dict returns:

{
    "content": [{"type": "text", "text": "Tool response"}],
    "role": "tool",
    "cache_control": {"type": "ephemeral"},
    "tool_call_id": "call_123",
    "name": "test_tool",
}

The returned message contains a top-level cache marker even though cache_enabled is False.

Steps to Reproduce
  1. Check out Software Agent SDK main at commit 007721b3d2bfccd1469f8008a514af13b7ae1b71.

  2. Add the following test to tests/sdk/llm/test_message.py:

def test_message_tool_role_omits_cache_prompt_when_cache_disabled():
    from openhands.sdk.llm.message import Message, TextContent

    message = Message(
        role="tool",
        content=[TextContent(text="Tool response", cache_prompt=True)],
        tool_call_id="call_123",
        name="test_tool",
    )

    result = message.to_chat_dict(
        **{
            **DEFAULT_SERIALIZATION_OPTS,
            "cache_enabled": False,
            "function_calling_enabled": True,
        }
    )

    assert "cache_control" not in result
    assert "cache_control" not in result["content"][0]
  1. Run:
uv run pytest tests/sdk/llm/test_message.py -q
  1. Observe that the newly added assertion fails while the 30 existing tests pass.
Acceptance Criteria
  • cache_enabled=False prevents list serialization from emitting message-level or content-level cache_control fields.
  • Disabling caching remains effective when list serialization is selected by function calling or vision support.
  • Existing cache_enabled=True behavior for tool and non-tool messages remains unchanged.
  • Regression coverage includes a tool message with cache_prompt=True, cache_enabled=False, and function_calling_enabled=True.
Installation Method

Source checkout; focused pytest reproduction after the documented development setup

If you selected "Other", please specify

No response

SDK Version

main@007721b3d2bfccd1469f8008a514af13b7ae1b71 (openhands-sdk 1.42.1)

Version Confirmation
  • I have confirmed this bug exists on the LATEST version of OpenHands SDK
Python Version

3.13.2

Model Name (if applicable)

Not applicable; reproduced by directly invoking deterministic message-serialization code

Operating System

MacOS

Logs and Error Messages
tests/sdk/llm/test_message.py .....F.........................            [100%]

FAILED tests/sdk/llm/test_message.py::test_message_tool_role_omits_cache_prompt_when_cache_disabled
AssertionError: assert 'cache_control' not in {'cache_control': {'type': 'ephemeral'}, 'content':
[{'text': 'Tool response', 'type': 'text'}], 'name': 'test_tool', 'role': 'tool', ...}

1 failed, 30 passed in 0.14s
Minimal Code Sample
from openhands.sdk.llm.message import Message, TextContent

message = Message(
    role="tool",
    content=[TextContent(text="Tool response", cache_prompt=True)],
    tool_call_id="call_123",
    name="test_tool",
)

result = message.to_chat_dict(
    cache_enabled=False,
    vision_enabled=False,
    function_calling_enabled=True,
    force_string_serializer=False,
    send_reasoning_content=False,
)

print(result)
# {
#     'content': [{'type': 'text', 'text': 'Tool response'}],
#     'role': 'tool',
#     'cache_control': {'type': 'ephemeral'},
#     'tool_call_id': 'call_123',
#     'name': 'test_tool',
# }
Screenshots and Additional Context

At the tested commit, to_chat_dict selects list serialization when any of caching, vision, or function calling is enabled:

if not force_string_serializer and (
    cache_enabled or vision_enabled or function_calling_enabled
):
    message_dict = self._list_serializer(vision_enabled=vision_enabled)

However, _list_serializer does not receive cache_enabled. It calls item.to_llm_dict(), which emits cache_control whenever the content object's cache_prompt field is true:

item_dicts = item.to_llm_dict()

For tool messages, it then unconditionally records that caching is requested and promotes the marker to the message level:

if self.role == "tool" and item.cache_prompt:
    role_tool_with_prompt_caching = True
    for d in item_dicts:
        d.pop("cache_control", None)

The public LLM formatting path passes the current cache setting into to_chat_dict. APIBasedCritic also calls this method with cache_enabled=False and function_calling_enabled=True. Consequently, a message already carrying cache_prompt=True can emit a cache marker through these serialization paths even though caching is explicitly disabled.

One possible fix direction is to pass cache_enabled into _list_serializer and use it to gate both content-level cache markers and the tool-message promotion logic. Existing behavior when cache_enabled=True should remain unchanged.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in openhands-sdk/openhands/sdk/llm/message.py by tracing Message.to_chat_dict into _list_serializer, then run the focused reproduction in tests/sdk/llm/test_message.py with pytest. Done means the tool-message case omits both cache_control fields when caching is disabled, while existing enabled-cache behavior and the surrounding tests remain passing.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design, testing-qa
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
86/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.