TypeScript SDK: readline splits valid JSONL tool events at U+2028/U+2029
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 125k
- Forks
- 19.4k
- PR merge metrics
- PR metrics pending
Description
Summary
The TypeScript SDK uses readline.createInterface({ input: child.stdout, crlfDelay: Infinity }) to frame CLI JSONL output, then calls JSON.parse on each line. With Bun 1.4.0/1.4.2, readline also splits on literal U+2028 and U+2029. Both characters are legal inside JSON strings, so a valid tool-result event can be split into invalid fragments and fail with Failed to parse item.
This is not necessarily a Bun-only compatibility bug: current Node v24.x/v26.x source also includes these separators in readline's line-ending regex. Node v22.x source does not. Node v24/v26 behavior was inspected in source, not runtime-tested here.
SDK versions inspected
Both published @openai/codex-sdk 0.144.1 and 0.154.0 use this framing logic:
- sdk/typescript/src/exec.ts: readline over CLI stdout
- sdk/typescript/src/thread.ts: JSON.parse each yielded line
Minimal reproduction
This isolates the same framing/parsing path without credentials, an API call, or private data:
// repro.mjs
import { Readable } from "node:stream";
import { createInterface } from "node:readline";
for (const separator of ["\u2028", "\u2029"]) {
const record = JSON.stringify({
type: "item.completed",
item: {
id: "item_1",
type: "command_execution",
command: "example",
aggregated_output: "alpha" + separator + "beta",
exit_code: 0,
status: "completed",
},
});
// The full record itself is valid JSON.
JSON.parse(record);
const rl = createInterface({
input: Readable.from([record + "\n"]),
crlfDelay: Infinity,
});
let lines = 0;
let failures = 0;
for await (const line of rl) {
lines++;
try {
JSON.parse(line);
} catch {
failures++;
}
}
console.log({
separator: separator.codePointAt(0).toString(16),
lines,
failures,
});
}
Run with bun repro.mjs or node repro.mjs.
Observed results
Local runtime tests on macOS arm64:
| Runtime | Lines per record | JSON parse failures |
|---|---|---|
| Bun 1.3.11 | 1 | 0 |
| Bun 1.4.0 | 2 | 2 |
| Bun 1.4.2 | 2 | 2 |
| Node v23.7.0 | 1 | 0 |
Also reproduced with an actual child_process stdout pipe on Bun 1.4.0. Escaping U+2028 as the six ASCII characters backslash-u-2028 avoids the problem. Ordinary text passes.
The first fragment ends inside aggregated_output and produces an unterminated-string error. This was encountered while consuming a tool result containing a Unicode line separator.
Expected behavior / proposed fix
Frame the CLI JSONL protocol explicitly on LF (byte 0x0A), rather than inheriting readline's broader text-line semantics.
Suggested implementation considerations:
- Preserve U+2028/U+2029 inside JSON strings.
- Buffer partial records across stdout chunks.
- Preserve UTF-8 characters split across chunks (e.g. decode complete byte records, or use a streaming decoder).
- Handle CRLF and final unterminated records deliberately, retaining the existing contract.
- Preserve subprocess cancellation, cleanup, and exit-error behavior.
- Keep genuinely malformed JSON as an error; do not suppress JSON.parse failures.
Suggested regression tests: literal U+2028/U+2029, multiple records in one chunk, one record across chunks, split multibyte UTF-8, cancellation/exit, and malformed input.
References
- https://unpkg.com/@openai/codex-sdk@0.154.0/dist/index.js
- https://github.com/oven-sh/bun/blob/bun-v1.4.2/src/js/internal/readline/interface.js#L65-L75
- https://github.com/nodejs/node/blob/v24.x/lib/internal/readline/interface.js
- https://github.com/nodejs/node/blob/v22.x/lib/internal/readline/interface.js
No credentials or private payloads are included.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with sdk/typescript/src/exec.ts, where readline frames CLI stdout, and sdk/typescript/src/thread.ts, where each line is parsed. Run repro.mjs under Bun and Node, then trace the existing subprocess tests and cancellation or exit handling. Done means literal U+2028/U+2029 remain within valid JSONL records, chunking and CRLF cases work, and malformed JSON still fails.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bun, node.js, typescript
- Domain
- developer-experience, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 65/100