cloudflare / cloudflare/workers-sdk

Miniflare doesn't support zstd compressed subrequests

Open
#9,522 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
TypeScript
Stars
4.5k
Forks
1.5k
Avg merge
3d 8h
Merged PRs (30d)
187

Description

### What versions & operating system are you using?

4.19.1

### Please provide a link to a minimal reproduction

_No response_

### Describe the Bug

Simple repro:

```
const responseStream = (await fetch('https://ping.igor-dev.workers.dev/', {headers: {'accept-encoding': 'zstd'}})).body;
```
If you read the stream now, you'll find that the stream is provided in its raw form rather than decompressed.

You'll also see that the response has the following header:
```
mf-content-encoding: zstd
```

which seems to come from: https://github.com/cloudflare/workers-sdk/blob/9d2b443a533d0532a87177b36b390a857ef3be04/packages/miniflare/src/index.ts#L2412-L2421

This issue seems very related to #8004 but with the difference that while gzip and brotli streams are decompressed automatically by miniflare (even when not needed during a proxy pass through as described in #8004).

Unlike with brotli or gzip, `zstd` compressed responses fail to be decompressed by miniflare/undici and can't be read within a worker. Additionally these `zstd` compressed responses can't be passed through the worker to the client because the `content-encoding` response header is modified from `content-encoding: zstd` to `mf-content-encoding: zstd` making the client believe that the content is not encoded/compressed.

### Please provide any relevant error logs

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by inspecting packages/miniflare/src/index.ts around lines 2412-2421 and compare its zstd handling with the gzip and brotli paths. Read issue #8004 for related proxy behavior, then reproduce the fetch with accept-encoding: zstd. Done means zstd responses can be read decompressed in a worker and passed through without the content-encoding header being misrepresented.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.