google-gemini / google-gemini/gemini-cli

DevTools HTTP stream corrupts UTF-8 characters split across chunks

Open Beginner friendly
#29,141 6 comments 0 reactions 0 assignees View on GitHub
area/core effort/small kind/bug priority/p2 status/bot-triaged
Dominant language
TypeScript
Stars
107k
Forks
14.6k
Avg merge
2d 3h
Merged PRs (30d)
45

Description

### What happened?

The Node HTTP interception path in `ActivityLogger.patchNodeHttp()` decodes each `data` chunk independently with `chunkBuffer.toString('utf8')`.

HTTP chunk boundaries are arbitrary, so they can split a multibyte UTF-8 character. The DevTools raw network view renders the emitted `chunk.data` values directly. When a response containing `€` is delivered as one byte followed by two bytes, the streamed activity text becomes `���`, while the completed response body (decoded after `Buffer.concat`) is correctly `€`.

This makes the live/raw activity stream disagree with the final response and corrupts non-ASCII response text whenever a code point crosses a Node stream chunk boundary.

#### Deterministic reproduction

Verified on current `main` at `0bd1d43`.

A focused Vitest probe replaces `http.request` with a local fake request, enables `ActivityLogger`, and emits this response:

```ts
const euro = Buffer.from('€');
response.emit('data', euro.subarray(0, 1));
response.emit('data', euro.subarray(1));
response.emit('end');

const events = logger.getBufferedLogs().network;
const streamed = events
.flatMap((event) => (event.chunk ? [event.chunk.data] : []))
.join('');
const completed = events.find((event) => event.response)?.response?.body;

expect(streamed).toBe('€');
expect(completed).toBe('€');
```

Observed failure:

```text
Expected: "€"
Received: "���"
```

The existing focused suite remains green:

```text
npm test -w @google/gemini-cli -- src/utils/activityLogger.test.ts
8 passed
```

### What did you expect to happen?

The per-response streaming decoder should retain incomplete UTF-8 sequences across `data` events and flush at `end`, so concatenating the emitted textual chunks produces the same decoded text as the completed uncompressed response body.

A per-response `StringDecoder('utf8')` or streaming `TextDecoder` would preserve chunk boundaries without changing the final response capture.

If maintainers consider this suitable for a community contribution, I can prepare a focused regression test and patch after approval.

### Client information

Client Information

- Source revision: `google-gemini/gemini-cli@0bd1d43`
- Package version: `0.59.0-nightly.20260825.g812f7a2bc`
- Platform: macOS
- Node.js: `v25.4.0`
- Reproduction is local and deterministic; it does not require a Gemini login, model call, or network service.

### Login information

Not applicable; the reproduction uses a mocked Node HTTP request and response.

### Anything else we need to know?

I searched open issues and pull requests for ActivityLogger, DevTools, HTTP chunk, multibyte, UTF-8, and replacement-character variants and found no duplicate. This report is limited to non-security DevTools/activity-log correctness.

Contributor guide

Open the contributing guide

Research direction

Start at the ActivityLogger.patchNodeHttp() entry point and the focused suite in src/utils/activityLogger.test.ts; reproduce the split-€ case described in the issue. Done means streamed chunk data preserves UTF-8 characters across data events, flushes at end, and matches the completed response body; run npm test -w @google/gemini-cli -- src/utils/activityLogger.test.ts.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
devtools
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
86/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.