nodejs / nodejs/node

stream: Writable.toWeb()/Duplex.toWeb() settles write() before a mutable chunk is consumed

Ouverte
#64,549 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Personne n'a encore pris cette issue.

Langage dominant
JavaScript
Étoiles
122k
Forks
37.3k
Merge moyen
4 j 2 h
PR mergées (30 j)
283

Description

Version

v24.15.0

Platform
Microsoft Windows NT 10.0.26200.0 x64
Subsystem

stream, webstreams

What steps will reproduce the bug?

When fs.WriteStream is converted using Writable.toWeb(), the promise returned
by writer.write() can settle before the file stream has consumed the supplied
Uint8Array.

This makes it unsafe to reuse a buffer after awaiting writer.write().

const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');
const { Writable } = require('node:stream');

(async () => {
  const file = path.join(
    os.tmpdir(),
    `node-writable-to-web-${process.pid}.bin`,
  );

  const fileStream = fs.createWriteStream(file);
  const writer = Writable.toWeb(fileStream).getWriter();

  // Reusable scratch buffer.
  const storage = new Uint8Array(4);

  storage.set([1, 2, 3, 4]);
  await writer.write(storage.subarray());

  // The previous write promise has settled, so reuse the buffer.
  storage.set([5, 6, 7, 8]);
  await writer.write(storage.subarray());

  await writer.close();

  const content = fs.readFileSync(file);
  console.log([...content]);

  fs.unlinkSync(file);
})().catch(console.error);

Run:

$ node repro.js
[
  5, 6, 7, 8,
  5, 6, 7, 8
]
How often does it reproduce? Is there a required condition?

It reproduces consistently when:

  1. A mutable Uint8Array or subarray() is written.
  2. The native writable.write() call returns true.
  3. The backing buffer is reused after the Web Streams write() promise settles.
  4. The native writable has not yet consumed the original bytes.

The problem can depend on chunk size. When the native write() returns false,
the adapter waits for backpressure, so larger chunks may appear to work.

The writable side returned by Duplex.toWeb() is affected by the same adapter
behavior.

What is the expected behavior? Why is that the expected behavior?

The file should contain the values supplied by the two completed writes:

[
  1, 2, 3, 4,
  5, 6, 7, 8
]

The producer mutates the buffer only after the first writer.write() promise
has settled.

The Web Streams specification advises producers not to mutate a mutable chunk
until the promise returned by write() settles:

https://streams.spec.whatwg.org/#default-writer-write

Following that guidance should ensure that the underlying sink processes the
same bytes that were passed to write().

What do you see instead?

The file contains the second value twice:

[
  5, 6, 7, 8,
  5, 6, 7, 8
]

The first writer.write() promise settles while fs.WriteStream still retains
a reference to the first subarray(). Reusing the backing buffer therefore also
changes the bytes of the pending file write.

Changing subarray() to slice() avoids the corruption because slice() copies
the bytes, but it requires an allocation and copy for every write.

What do you see instead?

The boolean returned by native writable.write() only represents backpressure.
It does not indicate that the specific chunk has finished being processed.

Node's native writable API provides a per-write callback:

writable.write(chunk, callback);

https://nodejs.org/api/stream.html#writablewritechunk-encoding-callback

A possible fix would be for the Web Streams adapter to:

  • Settle the Web write() promise from the native per-write callback.
  • If native write() returns false, also honor the drain event.
  • Reject the Web write promise if the callback reports an error.

This would allow callers to safely reuse mutable buffers after awaiting
writer.write() without requiring slice() copies.

Additional information

No response

Guide de contribution

Ouvrir le guide de contribution

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Piste de recherche

Commencez par les points d’entrée Writable.toWeb() et Duplex.toWeb() et suivez la manière dont native writable.write() mène à terme la promesse d’écriture de Web Streams. Reproduisez le problème avec le script fs.WriteStream fourni, puis ajoutez un test de régression pour la réutilisation d’un Uint8Array mutable ; le travail est terminé lorsque chaque écriture terminée conserve ses octets d’origine et que les erreurs de backpressure ou de callback continuent d’être gérées.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
javascript, node.js
Domaine
backend
Type d'issue
Bug
Difficulté
4/5
Temps estimé
3-5 jours
Activité
Calme
Clarté
Plutôt claire
Accessibilité débutants
48/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.