stream: Writable.toWeb()/Duplex.toWeb() settles write() before a mutable chunk is consumed
还没有人认领这个 Issue。
- 主要语言
- JavaScript
- 星标
- 122k
- 派生
- 37.4k
- 平均合并
- 4 天 3 小时
- 30 天内合并 PR
- 272
描述
Version
v24.15.0
Platform
Microsoft Windows NT 10.0.26200.0 x64
Subsystem
stream, webstreams
What steps will reproduce the bug?
When fs.WriteStream is converted using Writable.toWeb(), the promise returned
by writer.write() can settle before the file stream has consumed the supplied
Uint8Array.
This makes it unsafe to reuse a buffer after awaiting writer.write().
const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');
const { Writable } = require('node:stream');
(async () => {
const file = path.join(
os.tmpdir(),
`node-writable-to-web-${process.pid}.bin`,
);
const fileStream = fs.createWriteStream(file);
const writer = Writable.toWeb(fileStream).getWriter();
// Reusable scratch buffer.
const storage = new Uint8Array(4);
storage.set([1, 2, 3, 4]);
await writer.write(storage.subarray());
// The previous write promise has settled, so reuse the buffer.
storage.set([5, 6, 7, 8]);
await writer.write(storage.subarray());
await writer.close();
const content = fs.readFileSync(file);
console.log([...content]);
fs.unlinkSync(file);
})().catch(console.error);
Run:
$ node repro.js
[
5, 6, 7, 8,
5, 6, 7, 8
]
How often does it reproduce? Is there a required condition?
It reproduces consistently when:
- A mutable
Uint8Arrayorsubarray()is written. - The native
writable.write()call returnstrue. - The backing buffer is reused after the Web Streams
write()promise settles. - The native writable has not yet consumed the original bytes.
The problem can depend on chunk size. When the native write() returns false,
the adapter waits for backpressure, so larger chunks may appear to work.
The writable side returned by Duplex.toWeb() is affected by the same adapter
behavior.
What is the expected behavior? Why is that the expected behavior?
The file should contain the values supplied by the two completed writes:
[
1, 2, 3, 4,
5, 6, 7, 8
]
The producer mutates the buffer only after the first writer.write() promise
has settled.
The Web Streams specification advises producers not to mutate a mutable chunk
until the promise returned by write() settles:
https://streams.spec.whatwg.org/#default-writer-write
Following that guidance should ensure that the underlying sink processes the
same bytes that were passed to write().
What do you see instead?
The file contains the second value twice:
[
5, 6, 7, 8,
5, 6, 7, 8
]
The first writer.write() promise settles while fs.WriteStream still retains
a reference to the first subarray(). Reusing the backing buffer therefore also
changes the bytes of the pending file write.
Changing subarray() to slice() avoids the corruption because slice() copies
the bytes, but it requires an allocation and copy for every write.
What do you see instead?
The boolean returned by native writable.write() only represents backpressure.
It does not indicate that the specific chunk has finished being processed.
Node's native writable API provides a per-write callback:
writable.write(chunk, callback);
https://nodejs.org/api/stream.html#writablewritechunk-encoding-callback
A possible fix would be for the Web Streams adapter to:
- Settle the Web
write()promise from the native per-write callback. - If native
write()returnsfalse, also honor thedrainevent. - Reject the Web write promise if the callback reports an error.
This would allow callers to safely reuse mutable buffers after awaiting
writer.write() without requiring slice() copies.
Additional information
No response
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
调研方向
从 Writable.toWeb() 和 Duplex.toWeb() 入口开始,跟踪 native writable.write() 如何完成 Web Streams 的写入 promise。使用提供的 fs.WriteStream 脚本重现该问题,然后为重用可变 Uint8Array 添加回归测试;完成标准是每次已完成的写入都保留其原始字节,并且 backpressure 或 callback 错误仍能得到处理。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- javascript, node.js
- 领域
- backend
- Issue 类型
- 缺陷
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 活跃度
- 冷清
- 描述清晰度
- 基本清楚
- 新手友好度
- 48/100