cloudflare / cloudflare/workerd
Buffer.prototype.utf8Write` (and all other `*Write` methods) throw `ERR_OUT_OF_RANGE` when `length` is omitted and `offset > 0`
- Dominant language
- C++
- Stars
- 8.7k
- Forks
- 739
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 174
Description
## Summary
Under `nodejs_compat`, the encoding-specific write methods (`utf8Write`, `asciiWrite`, `latin1Write`, `ucs2Write`, `hexWrite`, `base64Write`, `base64urlWrite`) default an omitted `length` to **the full buffer size** instead of **the remaining bytes after `offset`**, then validate it against the remaining bytes. As a result, any two-argument call with a non-zero offset — `buf.utf8Write(str, pos)` — always throws, regardless of the string being written:
```
RangeError [ERR_OUT_OF_RANGE]: The value of "length" is out of range. It must be >= 0 && <= 9. Received 10
```
In Node.js the same call succeeds: `length` defaults to `buf.length - offset` and the write is clamped. (Node itself briefly shipped this exact bug class in v22.7.0 — nodejs/node#54523 — and reverted it in v22.8.0 because it broke Firestore, protobufjs, and others.)
## Minimal reproduction
```js
import { Buffer } from 'node:buffer';
export default {
fetch() {
const buf = Buffer.alloc(10);
// Node.js: writes 5 bytes at offset 1, returns 5.
// workerd: throws ERR_OUT_OF_RANGE before writing anything.
const written = buf.utf8Write('hello', 1);
return new Response(`wrote ${written}`);
},
};
```
with `compatibility_flags = ["nodejs_compat"]`. Any `offset > 0` with `length` omitted throws; `offset = 0` works because `length = this.length` then passes validation.
## Root cause
In [`src/node/internal/internal_buffer.ts`](https://github.com/cloudflare/workerd/blob/main/src/node/internal/internal_buffer.ts#L1024-L1040):
```ts
Buffer.prototype.utf8Write = function utf8Write(string, offset, length) {
offset ??= 0;
length ??= this.length; // ← should be `this.length - offset` (Node semantics)
validateOffset(offset as number, 'offset', 0, this.length);
validateOffset(length as number, 'length', 0, this.length - offset); // ← always fails when offset > 0
...
};
```
The same `length ??= this.length` + `validateOffset(length, 'length', 0, this.length - offset)` pattern is repeated in all seven `*Write` methods (lines ~916–1040). The generic `Buffer.prototype.write` just below handles the omitted-`length` case correctly, so only these fast-path methods are affected.
## Real-world impact
protobufjs ≥ 7.6.0 changed its `Buffer` detection from a hidden `require("buffer")` (which fails in worker bundles, falling back to a pure-JS Uint8Array writer) to reading `globalThis.Buffer` — which exists under `nodejs_compat`. Its `BufferWriter` encodes every string field of 40+ characters via `buf.utf8Write(val, pos)` ([writer_buffer.js](https://github.com/protobufjs/protobuf.js/blob/master/src/writer_buffer.js)), where `pos > 0` for any string field that isn't the very first bytes of the message. So on workerd, **protobuf-encoding almost any real message with protobufjs ≥ 7.6 throws**, e.g.:
```
RangeError: The value of "length" is out of range. It must be >= 0 && <= 14041. Received 14109
```
This took down our (GitBook's) document-import pipeline after a routine protobufjs security bump. Given Node's 22.7.0 experience, anything else in the ecosystem calling `utf8Write(string, offset)` is affected the same way.
## Expected behavior
Match Node.js: when `length` is omitted, default it to `this.length - offset` (and clamp the write), for all `*Write` methods.
## Workaround
Force protobufjs onto its portable writer: `protobuf.util.Buffer = null; protobuf.configure();` — or avoid calling `*Write` methods without an explicit `length`.
Contributor guide
Research direction
Start in src/node/internal/internal_buffer.ts around lines 916–1040 and inspect the seven encoding-specific *Write methods, then compare their omitted-length handling with Buffer.prototype.write just below. Done means omitted length uses the remaining capacity after offset, matches Node.js behavior, and no longer raises ERR_OUT_OF_RANGE for valid two-argument calls.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, typescript
- Domain
- api
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100