openai / openai/codex

TypeScript SDK: readline splits valid JSONL tool events at U+2028/U+2029

Open
#45,727 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

app-server bug sdk
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

Summary

The TypeScript SDK uses readline.createInterface({ input: child.stdout, crlfDelay: Infinity }) to frame CLI JSONL output, then calls JSON.parse on each line. With Bun 1.4.0/1.4.2, readline also splits on literal U+2028 and U+2029. Both characters are legal inside JSON strings, so a valid tool-result event can be split into invalid fragments and fail with Failed to parse item.

This is not necessarily a Bun-only compatibility bug: current Node v24.x/v26.x source also includes these separators in readline's line-ending regex. Node v22.x source does not. Node v24/v26 behavior was inspected in source, not runtime-tested here.

SDK versions inspected

Both published @openai/codex-sdk 0.144.1 and 0.154.0 use this framing logic:

  • sdk/typescript/src/exec.ts: readline over CLI stdout
  • sdk/typescript/src/thread.ts: JSON.parse each yielded line

Minimal reproduction

This isolates the same framing/parsing path without credentials, an API call, or private data:

// repro.mjs
import { Readable } from "node:stream";
import { createInterface } from "node:readline";

for (const separator of ["\u2028", "\u2029"]) {
  const record = JSON.stringify({
    type: "item.completed",
    item: {
      id: "item_1",
      type: "command_execution",
      command: "example",
      aggregated_output: "alpha" + separator + "beta",
      exit_code: 0,
      status: "completed",
    },
  });

  // The full record itself is valid JSON.
  JSON.parse(record);

  const rl = createInterface({
    input: Readable.from([record + "\n"]),
    crlfDelay: Infinity,
  });

  let lines = 0;
  let failures = 0;
  for await (const line of rl) {
    lines++;
    try {
      JSON.parse(line);
    } catch {
      failures++;
    }
  }
  console.log({
    separator: separator.codePointAt(0).toString(16),
    lines,
    failures,
  });
}

Run with bun repro.mjs or node repro.mjs.

Observed results

Local runtime tests on macOS arm64:

Runtime Lines per record JSON parse failures
Bun 1.3.11 1 0
Bun 1.4.0 2 2
Bun 1.4.2 2 2
Node v23.7.0 1 0

Also reproduced with an actual child_process stdout pipe on Bun 1.4.0. Escaping U+2028 as the six ASCII characters backslash-u-2028 avoids the problem. Ordinary text passes.

The first fragment ends inside aggregated_output and produces an unterminated-string error. This was encountered while consuming a tool result containing a Unicode line separator.

Expected behavior / proposed fix

Frame the CLI JSONL protocol explicitly on LF (byte 0x0A), rather than inheriting readline's broader text-line semantics.

Suggested implementation considerations:

  • Preserve U+2028/U+2029 inside JSON strings.
  • Buffer partial records across stdout chunks.
  • Preserve UTF-8 characters split across chunks (e.g. decode complete byte records, or use a streaming decoder).
  • Handle CRLF and final unterminated records deliberately, retaining the existing contract.
  • Preserve subprocess cancellation, cleanup, and exit-error behavior.
  • Keep genuinely malformed JSON as an error; do not suppress JSON.parse failures.

Suggested regression tests: literal U+2028/U+2029, multiple records in one chunk, one record across chunks, split multibyte UTF-8, cancellation/exit, and malformed input.

References

No credentials or private payloads are included.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with sdk/typescript/src/exec.ts, where readline frames CLI stdout, and sdk/typescript/src/thread.ts, where each line is parsed. Run repro.mjs under Bun and Node, then trace the existing subprocess tests and cancellation or exit handling. Done means literal U+2028/U+2029 remain within valid JSONL records, chunking and CRLF cases work, and malformed JSON still fails.

Written by the indexing model from the issue text.

Assessment

Tech stack
bun, node.js, typescript
Domain
developer-experience, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.