ClickHouse / ClickHouse/clickhouse-js
In-band mid-stream exceptions are not detected for exec() streams (Arrow/Native) or ResultSet.text(): callers get a raw ECONNRESET "aborted" instead of the server error
- Dominant language
- TypeScript
- Stars
- 331
- Forks
- 74
- PR merge metrics
- No merged PRs in 30d
Description
## Description
When a query fails **after** ClickHouse has already committed `200 OK` and started streaming the body (e.g. a runtime `throwIf` partway through a large result), the server (25.11+) appends an in-band exception block, announced by the `x-clickhouse-exception-tag` response header and delimited by `__exception__` markers.
`@clickhouse/client` only inspects that in-band block in **one** place: the newline-splitting transform inside `ResultSet.stream()` (`packages/client-node/src/result_set.ts:252-257`, `packages/client-web/src/result_set.ts:200-205`), which calls `extractErrorAtTheEndOfChunk`. Every other way of consuming a response body skips it:
1. **`client.exec()`** — this is the documented way to fetch binary/columnar output (see [`examples/node/performance/select_parquet_as_file.ts`](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/performance/select_parquet_as_file.ts), which pipes the `exec()` stream straight to a file). `exec()` returns the transport stream untouched — nothing ever looks at `x-clickhouse-exception-tag`, even though the header is present on the response. A mid-stream failure surfaces as `Error: aborted` / `ECONNRESET`, and any already-written output file silently contains a truncated payload plus the raw `__exception__` trailer bytes.
2. **`ResultSet.text()`** — reads the whole body via `getAsText()` with no exception check, so the same query that produces a clean `ClickHouseError` through `.stream()` produces `Error: aborted` (`ECONNRESET`) through `.text()`.
This is the Node.js/Web analogue of https://github.com/ClickHouse/clickhouse-connect/issues/913 (Arrow streaming methods bypassing in-band exception detection).
Note: streaming formats that ClickHouse buffers fully server-side (e.g. `Parquet`) are *not* affected — the error arrives before any body bytes, so the normal error path fires. The problem is specific to formats that actually stream (`Arrow`, `Native`, `RowBinary`, `CSV`, ...).
## ClickHouse server version
`26.7.1.1315` (local, verified end-to-end).
## Reproduction
`packages/client-node/__tests__/integration/inband_exception_exec.test.ts`:
```ts
import { describe, it, expect, beforeEach, afterEach } from "vitest";
import type { ClickHouseClient } from "@clickhouse/client-common";
import { createTestClient } from "@test/utils";
import type Stream from "stream";
const QUERY =
"SELECT number, throwIf(number = 1000000, 'boom') FROM system.numbers";
describe("in-band exception on non-row-parser paths", () => {
let client: ClickHouseClient;
beforeEach(() => {
client = createTestClient() as ClickHouseClient;
});
afterEach(async () => {
await client.close();
});
// PASSES today - the row parser detects the in-band exception.
it("stream() surfaces the server error", async () => {
const rs = await client.query({ query: QUERY, format: "JSONEachRow" });
await expect(
(async () => {
for await (const rows of rs.stream()) void rows;
})(),
).rejects.toMatchObject({ code: "395" });
});
// FAILS today.
it("exec() with FORMAT Arrow surfaces the server error", async () => {
const { stream, response_headers } = await client.exec({
query: `${QUERY} FORMAT Arrow`,
});
expect(response_headers["x-clickhouse-exception-tag"]).toBeDefined(); // header IS there
await expect(
(async () => {
for await (const chunk of stream) void chunk;
})(),
).rejects.toMatchObject({ code: "395" });
});
// FAILS today.
it("text() surfaces the server error", async () => {
const rs = await client.query({ query: QUERY, format: "CSV" });
await expect(rs.text()).rejects.toMatchObject({ code: "395" });
});
});
```
Observed (instrumented run against 26.7.1.1315):
```
JSONEachRow stream(): ClickHouseError code=395 "boom: while executing 'FUNCTION throwIf(...)'" <- correct
exec() FORMAT Arrow: 3934840 bytes consumed, then Error code=ECONNRESET msg="aborted" <- no server error
exec() FORMAT Native: 8831550 bytes consumed, then Error code=ECONNRESET msg="aborted" <- no server error
CSV text(): Error code=ECONNRESET msg="aborted" <- no server error
exec() FORMAT Parquet: 0 bytes, ClickHouseError code=395 <- fine (buffered server-side)
```
Expected: all four raise `ClickHouseError` with `code: '395'` and the `boom` message, matching the `stream()` path.
## Suggested fix
The building blocks already exist — `extractErrorAtTheEndOfChunk` and `EXCEPTION_TAG_HEADER_NAME` in `packages/client-common/src/utils/stream.ts` — they are just not applied outside the row parser:
- Wire a format-agnostic in-band exception detector into the stream returned by `ClickHouseClient.exec()` when `x-clickhouse-exception-tag` is present on the response (`packages/client-common/src/client.ts:420`, plus the node/web connection `exec` implementations). The passthrough detector proposed in #778 (`RawStreamExceptionDetector`) looks like the right primitive; #778 only wires it into `ResultSet.binaryStream()` for `query()` results, so `exec()` would still be uncovered.
- Apply the same check in `ResultSet.text()` (`packages/client-node/src/result_set.ts:157`, `packages/client-web/src/result_set.ts`) — currently `getAsText()` bypasses it entirely.
Related but not covering these paths: #778 (`binaryStream()` for `query()`), #803 (opt-in transformer for the Node `ResultSet` row parser), #974 (false positives in the current `\r\n` heuristic).
## Link
Relayed from https://github.com/ClickHouse/clickhouse-connect/issues/913
Central tracking issue: https://github.com/ClickHouse/integrations-ai-playground/issues/330
Contributor guide
Research direction
Start with packages/client-common/src/utils/stream.ts and the exec implementations referenced around packages/client-common/src/client.ts:420, then compare the existing ResultSet.stream() handling. Inspect packages/client-node/src/result_set.ts:157 and its web counterpart, and run packages/client-node/__tests__/integration/inband_exception_exec.test.ts. Done means exec() and ResultSet.text() surface ClickHouseError code 395 for the streaming reproductions without regressing the existing stream() path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- nodejs, typescript
- Domain
- api, backend, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100