cloudflare / cloudflare/workers-sdk
Miniflare doesn't support zstd compressed subrequests
- Dominant language
- TypeScript
- Stars
- 4.5k
- Forks
- 1.5k
- Avg merge
- 3d 8h
- Merged PRs (30d)
- 187
Description
### What versions & operating system are you using?
4.19.1
### Please provide a link to a minimal reproduction
_No response_
### Describe the Bug
Simple repro:
```
const responseStream = (await fetch('https://ping.igor-dev.workers.dev/', {headers: {'accept-encoding': 'zstd'}})).body;
```
If you read the stream now, you'll find that the stream is provided in its raw form rather than decompressed.
You'll also see that the response has the following header:
```
mf-content-encoding: zstd
```
which seems to come from: https://github.com/cloudflare/workers-sdk/blob/9d2b443a533d0532a87177b36b390a857ef3be04/packages/miniflare/src/index.ts#L2412-L2421
This issue seems very related to #8004 but with the difference that while gzip and brotli streams are decompressed automatically by miniflare (even when not needed during a proxy pass through as described in #8004).
Unlike with brotli or gzip, `zstd` compressed responses fail to be decompressed by miniflare/undici and can't be read within a worker. Additionally these `zstd` compressed responses can't be passed through the worker to the client because the `content-encoding` response header is modified from `content-encoding: zstd` to `mf-content-encoding: zstd` making the client believe that the content is not encoded/compressed.
### Please provide any relevant error logs
_No response_
Contributor guide
Research direction
Start by inspecting packages/miniflare/src/index.ts around lines 2412-2421 and compare its zstd handling with the gzip and brotli paths. Read issue #8004 for related proxy behavior, then reproduce the fetch with accept-encoding: zstd. Done means zstd responses can be read decompressed in a worker and passed through without the content-encoding header being misrepresented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100