lovell / lovell/sharp

Enhancement: First-class support for Web Streams API

Open
#4,234 14 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
JavaScript
Stars
32.7k
Forks
1.4k
Avg merge
1d 14h
Merged PRs (30d)
5

Description

## Question about an existing feature

### What are you trying to achieve?

I'm trying to convert a `Sharp` object as returned by `sharp("...").webp()` to a [web standard `ReadableStream`](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream) for compatibility with [Deno.serve()](https://docs.deno.com/api/deno/~/Deno.serve) / [`new Response(body)`](https://developer.mozilla.org/en-US/docs/Web/API/Response/Response#parameters)

### When you searched for similar issues, what did you find that might be related?

I have searched a lot, e.g. in the "issues" tab on this repo, but couldn't find any truly related issue. (Aside from #4013 maybe)
I managed to make it work with:
- [`new ReadableStream`](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream/ReadableStream): `new ReadableStream({ start(controller) { sharpObject.on("data", chunk => controller.enqueue(chunk) } })` - but it's slightly slower than the classic `sharpObject.pipe(res)` you'd use with `http.createServer` or express
- [`Readable.toWeb(sharpObject)`](https://nodejs.org/api/stream.html#streamreadabletowebstreamreadable-options) - appears to pipe some image data, but the served image appears to be broken/blank and it's significantly slower (~3x) on a 20MB input file than `.pipe(res)` or the previous `new ReadableStream` option
*(Note: `toWeb` was added in Node v17 and is still marked as experimental)*
- [`createReadableStreamFromReadable` from @remix-run/node](https://github.com/remix-run/remix/blob/cab5a7c09c3496d253de29637e73f859a1c72776/packages/remix-node/stream.ts#L66) works but is ~3x as slow as `.pipe(res)`. They also use their own `new ReadableStream`. This function can be used with `import { createReadableStreamFromReadable } from "@remix-run/node"` even when not using the Remix framework.

Somewhat related: Stackoverflow posts about the difference between web standard ReadableStream and Node Readable
- https://stackoverflow.com/a/66629140/9948553
- https://stackoverflow.com/q/37614649/9948553

### Please provide a minimal, standalone code sample, without other dependencies, that demonstrates this question

```ts
const sharpObject = sharp("img.png").webp();
const readableStream: ReadableStream = sharpObject;
/**
* TypeScript will already warn here that
* ` 'Sharp' is missing the following properties from type 'ReadableStream' `,
* which at runtime holds true as the `readableStream` variable doesn't work
* with e.g. `new Response(readableStream)` in Deno
*/
```

### Please provide sample image(s) that help explain this question

independent of image file

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start from the sharp("img.png").webp() entry point and its existing Node Readable behavior, then compare it with Readable.toWeb and the Web Streams API examples in the issue. Done means a Sharp result can be used as a standard ReadableStream with Deno.serve() or new Response(body), without the broken output or major slowdown described.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, nodejs, typescript
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.