pingdotgg / pingdotgg/t3code

Usage scan crashes server on oversized Codex JSONL record

Open
#7,258 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
23k
Forks
5.9k
Avg merge
11h 14m
Merged PRs (30d)
357

Description

Summary

Opening the Usage page can terminate the entire t3@latest Node process when a Codex session rollout contains one very large JSONL record.

In the observed case:

  • 7 days worked.
  • 30 days and 90 days showed zero usage and then crashed the server.
  • The longer windows included an older Codex rollout with one 516.47 MiB line.
  • That line was a non-usage event_msg -> patch_apply_end -> delete record.
  • Removing only that irrelevant record made both 30-day and 90-day scans complete successfully.

Environment

  • Windows
  • Node.js v24.15.0
  • t3@latest, launched with the local web server
  • Codex transcripts under the normal .codex/sessions directory

Crash

node:internal/readline/interface:652
        this[kLine_buffer] += string;
                              ^

RangeError: Invalid string length
    at [_normalWrite] [as _normalWrite] (node:internal/readline/interface:652:31)
    at ReadStream.ondata (node:internal/readline/interface:263:23)
    at ReadStream.emit (node:events:509:28)
    at addChunk (node:internal/streams/readable:563:12)
    at readableAddChunkPushByteMode (node:internal/streams/readable:514:3)
    at Readable.push (node:internal/streams/readable:394:5)
    at node:internal/fs/streams:293:14
    at FSReqCallback.wrapper [as oncomplete] (node:fs:670:5)

Root cause

apps/server/src/usage/usageTranscriptReader.ts reads each transcript with:

NodeReadline.createInterface({
  input: NodeFS.createReadStream(filePath, { encoding: "utf8" }),
  crlfDelay: Infinity,
});

Node's readline accumulates an entire line into a JavaScript string before T3 can apply mightCarryUsage. A sufficiently large record exceeds V8's maximum string length and throws from the stream's data callback. The surrounding try/catch does not prevent the process-level crash.

Measured transcript:

  • Total file size before workaround: 551,526,950 bytes
  • JSONL records: 3,403
  • Oversized record: line 3,387
  • Oversized record length: 541,561,318 bytes
  • Record kind: event_msg -> patch_apply_end -> delete
  • File remained correctly newline-delimited

Workaround validation

After preserving the original transcript and removing only that non-usage line:

  • Active transcript size: 9,965,632 bytes
  • Remaining records: 3,402
  • Largest remaining line: 106,482 bytes
  • 30-day scan: 753 files, 1,559,096,780 bytes, 32,808 usage records, 0 failed files
  • 90-day scan: 753 files, 1,559,126,792 bytes, 32,811 usage records, 0 failed files

Both scans used T3 Code's current listTranscriptFiles and readTranscriptRecords implementation directly.

Expected behavior

An oversized or malformed transcript record should not terminate the T3 server. The Usage response should either:

  1. skip an oversized record that cannot affect usage, or
  2. skip/report the affected transcript while returning partial usage.

Suggested direction

Use a bounded byte/chunk line reader instead of unbounded Node readline. It should stop buffering a record after a conservative limit, drain through its newline without decoding the whole record, and continue scanning later records.

A focused test could expose an injectable low line limit so the regression can be covered without committing a hundreds-of-megabytes fixture.

Possibly related but not equivalent: #996 reports crashes around a heavy thread, without this Usage-page stack or transcript-reader path.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read apps/server/src/usage/usageTranscriptReader.ts and trace readTranscriptRecords from listTranscriptFiles; first inspect how the current readline-based scan handles the oversized JSONL record. Done means an oversized or malformed record no longer terminates the server, later records can still be scanned, and a focused test covers the behavior with a small injectable line limit.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
64/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.