anomalyco / anomalyco/opencode

webfetch always decodes as UTF-8, mojibake on non-UTF-8 sites

Open
#45,902 1 comment 0 reactions 1 assignee View on GitHub

@jlongster is already working on this.

Since Aug 28, 2026.

Dominant language
TypeScript
Stars
209k
Forks
27.5k
PR merge metrics
PR metrics pending

Description

Describe the bug

webfetch decodes the response body with new TextDecoder(), which is always UTF-8. The charset parameter from the Content-Type header is read into contentType but never used for decoding, and HTML <meta charset> is ignored entirely.

Any page served as GBK/GB18030, Shift_JIS, ISO-8859-1, etc. comes back as U+FFFD replacement-character mojibake, so text/markdown extraction produces garbage for a large class of CJK and legacy sites.

Steps to reproduce

Fetch a URL whose server returns Content-Type: text/plain; charset=gbk with a GBK-encoded body. The tool returns -filled text instead of the decoded characters.

Affected code

packages/core/src/tool/webfetch.tsconst content = new TextDecoder().decode(body).

Suggested fix

Extract charset from the Content-Type header, fall back to the HTML <meta charset> when the header omits it, then fall back to UTF-8 (and guard unsupported labels).

Environment
  • opencode version: latest dev

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.