OpenHands / OpenHands/software-agent-sdk

[Bug]: Gemini edit corrupts non-UTF-8 content during unrelated edits

Open
#5,108 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug priority:medium ready-for-dev release-note-required tools
Dominant language
Python
Stars
1.1k
Forks
539
Avg merge
1d 19h
Merged PRs (30d)
137

Description

Is there an existing issue for the same bug?
  • I have searched existing issues and this is not a duplicate.
Bug Description

The Gemini edit tool can change unrelated bytes when editing a file that is not valid UTF-8. It reads the file with replacement decoding, applies the requested edit, then writes everything back as UTF-8. For example, changing False to True in a Latin-1 file also changes café to caf�, but the tool reports success.

Expected Behavior

If the file cannot be decoded as UTF-8, return an error and leave its bytes unchanged.

Actual Behavior

Running uv run python reproduce.py with the script below reports success, but replaces undecodable bytes with U+FFFD when the file is written back.

Steps to Reproduce

From a checkout of release v1.49.0, run uv sync --frozen --dev. Save this as reproduce.py and run uv run python reproduce.py. It uses the real executor and a temporary file; no model or API key is needed.

from pathlib import Path
from tempfile import TemporaryDirectory

from openhands.tools.gemini.edit.definition import EditAction
from openhands.tools.gemini.edit.impl import EditExecutor

with TemporaryDirectory() as workspace:
    path = Path(workspace) / "settings.txt"
    original = b"caption = caf\xe9\nenabled = False\n"
    path.write_bytes(original)

    result = EditExecutor(workspace)(
        EditAction(
            file_path="settings.txt", old_string="False", new_string="True"
        )
    )

    print("is_error:", result.is_error)
    print("replacements_made:", result.replacements_made)
    print("before:", original)
    print("after:", path.read_bytes())
    print("bytes_preserved:", path.read_bytes() == original)
Acceptance Criteria
  • An edit on a file containing invalid UTF-8 returns an error without changing any bytes.
  • Valid UTF-8 edits still work, including files containing an actual U+FFFD character.
  • Existing file creation and replacement-count behavior still passes its tests.
Installation Method

Source checkout, UV_FROZEN=true make build (uv sync --dev using the repository lockfile).

If you selected "Other", please specify

Not applicable.

SDK Version

openhands-sdk==1.49.0, openhands-tools==1.49.0; reproduced on main at 97dbce5c37865dd7a3020d2b9662f4a76d11fa71 (release v1.49.0).

Version Confirmation
  • I have confirmed this bug exists on the LATEST version of OpenHands SDK.
Python Version

3.13.15 and 3.12.13.

Model Name (if applicable)

Not applicable. Direct tool execution without a model.

Operating System

macOS 26.6.2, arm64.

Logs and Error Messages
is_error: False
replacements_made: 1
before: b'caption = caf\xe9\nenabled = False\n'
after: b'caption = caf\xef\xbf\xbd\nenabled = True\n'
bytes_preserved: False
Screenshots and Additional Context

This report is limited to the Gemini edit tool. Rejecting invalid UTF-8 before writing would prevent the data loss; adding support for other encodings is separate work. I have a small fix and regression tests ready.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with openhands/tools/gemini/edit/impl.py and the EditExecutor entry point, then run the provided reproduce.py script after installing the locked dependencies. Add regression coverage for invalid UTF-8, valid UTF-8 containing U+FFFD, file creation, and replacement counts; done means invalid bytes remain unchanged while existing valid edits still pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.