nodejs / nodejs/node

fs module should support writing from Blob and web ReadableStream sources

Offen
#61,684 4 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Dieses Issue hat noch niemand übernommen.

feature request
Vorherrschende Sprache
JavaScript
Sterne
122k
Forks
37.3k
Ø Merge
4 T. 2 Std.
Gemergte PRs (30 T.)
283

Beschreibung

What is the problem this feature will solve?

Currently, all Node.js fs module methods for writing data (such as fs.write, fs.createWriteStream) only accept Buffers or strings. There is no built-in support for writing data directly from Blob or web ReadableStream sources. This limits interoperability with web platform APIs and makes working with large files or streaming data @less seamless, especially when trying to move data efficiently from web contexts into the filesystem. Furthermore, Node.js provides fs.readableWebStream, but no corresponding fs.createWritableStream that would allow writing from web streams or Blobs directly.

What is the feature you are proposing to solve the problem?

Extend the Node.js fs module with the following capabilities:

  • Allow fs.write and related write methods to accept Blob objects as a valid input, making it possible to write the entirety of a Blob's contents to a file without manual conversion to Buffer.
  • Introduce fs.createWritableStream (web-compatible), which would be a direct counterpart to fs.readableWebStream and support writing from web ReadableStream sources to files.

This would improve ease-of-use when working with large binary/web platform data and make fs more aligned with modern web standards.

What alternatives have you considered?

Current workarounds involve manually converting Blobs and web streams into Buffers or converting web streams into node:streams

The only solution we have today is to use the fs.promise.writeFile(path, webStream) which luckily supports async iterable

with that in mind, the NodeJS document for...

... filehandle.appendFile(data[, options]) says it support:

data: <string> | <Buffer> | <TypedArray> | <DataView> | <AsyncIterable> | <Iterable> | <Stream>

it should also say

data: <string> | <Buffer> | <TypedArray> | <DataView> | <AsyncIterable> | <Iterable> | <Stream> | <ReadableStream>

or just

data: <string> | <Buffer> | <TypedArray> | <DataView> | <AsyncIterable> | <Iterable>

as stream's are async iterable

and i also do not thing it should say Buffer anymore... buffer is a typed array already... so it can just say:

data: <string> | <TypedArray> | <DataView> | <AsyncIterable> | <Iterable>

but this is just only off-topic


able to write Blob's would enable efficient native copy operations (like using openAsBlob(path) and writing directly to another file) without data passing through the JS runtime.

able to write blobs avoids extra conversions/allocations in JS, reduces memory footprint and GC pressure


the solution i'm looking for is something like:

blob = new Blob(['hello world'])
await filehandle.write(blob)

await blob.stream().pipeTo(fs.createWritableStream(dest))

// using fetch is becoming more popular, so a web writable stream is highly requested
await new Response('...').body.pipeTo(fs.createWritableStream(dest))

Beitragsleitfaden

Beitragsleitfaden öffnen

Erste Schritte

  1. Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
  3. Forke das Repository und arbeite in einem Branch.
  4. Öffne einen Pull Request, der die Issue-Nummer nennt.

Rechercherichtung

Lies zuerst das Verhalten von fs.write, fs.promises.writeFile, fs.readableWebStream und filehandle.appendFile und vergleiche es anschließend mit den Blob- und web ReadableStream-Fällen im Issue. Als erledigt gilt, wenn der unterstützte Umfang abgestimmt ist und die angeforderten Schreib-APIs die genannten Blob- und Web-Stream-Anwendungsfälle verarbeiten.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
javascript, nodejs
Bereich
operating-systems
Issue-Typ
Feature
Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Aktivitätsstatus
Ruhig
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
42/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.