anomalyco / anomalyco/opencode

edit and apply_patch silently corrupt non-UTF-8 files (U+FFFD replacement)

Open
#45,924 2 comments 0 reactions 1 assignee View on GitHub

@neriousy is already working on this.

Since Aug 28, 2026.

Dominant language
TypeScript
Stars
209k
Forks
27.5k
PR merge metrics
PR metrics pending

Description

Describe the bug

edit and apply_patch decode file bytes with a non-fatal UTF-8 decoder (new TextDecoder("utf-8") and new TextDecoder("utf-8", { ignoreBOM: true })). Invalid sequences are silently replaced with U+FFFD.

When oldString matches an ASCII region of a file that contains non-UTF-8 bytes elsewhere (GBK/Latin-1 comments, legacy strings, stray bytes), the tool reports success and writes the entire file back as U+FFFD, permanently destroying every non-UTF-8 byte.

The read tool already rejects this with MalformedUtf8Error (File is not valid UTF-8) via a fatal decoder, but edit/apply_patch don't — so a file that read refuses is silently corrupted by edit.

Steps to reproduce
  1. Create a file with bytes const x = 1\n followed by GBK bytes d6 d0 ce c4 (中文).
  2. edit oldString: "const x = 1"newString: "const y = 1".
  3. The file is written back with the GBK bytes replaced by U+FFFD, and the tool reports success.
Affected code
  • packages/core/src/tool/edit.tsdecodeUtf8 uses new TextDecoder()
  • packages/core/src/tool/apply-patch.tsnew TextDecoder("utf-8", { ignoreBOM: true })
Suggested fix

Decode with { fatal: true } and fail with an explicit "file is not valid UTF-8" error, leaving the file untouched (matching read).

Environment
  • opencode version: latest dev

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.