dotnet / dotnet/runtime

[API Proposal]: Add support for Brotli Large Window decompression

Open
#133,018 3 comments 0 reactions 1 assignee Claimed by @iremyux View on GitHub
api-suggestion area-System.IO.Compression
Dominant language
C#
Stars
18.3k
Forks
5.6k
PR merge metrics
PR metrics pending

Description

### Background and motivation

## Background and motivation

`System.IO.Compression.BrotliStream` and `BrotliDecoder` currently support Brotli streams using the RFC 7932 window range, with a maximum window size of 2^24 bytes.

The upstream Brotli implementation additionally supports the Brotli Large Window extension, allowing window sizes up to 2^30 bytes. In the reference implementation this can be enabled using `BROTLI_DECODER_PARAM_LARGE_WINDOW`.

There is currently no public .NET API to opt into this decoding mode.

This has become relevant for PDF processing. The PDF Association has standardized a `BrotliDecode` filter for PDF which requires support for Brotli Large Window streams. A PDF parser relying on `System.IO.Compression.BrotliStream` therefore cannot fully implement the specification.

I verified the current limitation using a Brotli stream generated by the reference implementation with:

```text
--large_window=30
```

The reference Brotli decoder successfully decodes the stream when Large Window mode is enabled, while `System.IO.Compression.BrotliStream` rejects it.

PDF libraries would otherwise need to either:

* ship a separate Brotli decoder;
* introduce platform-specific native Brotli binaries; or
* intentionally not support part of the PDF Brotli specification.

It would therefore be useful to expose Large Window decompression through `System.IO.Compression`.

The existing behavior should remain unchanged by default. Large Window decoding should require an explicit opt-in, since accepting larger windows can significantly increase decoder memory requirements.

### API Proposal

## API Proposal

Introduce decompression options for Brotli, similar to the options pattern used by other compression APIs.

```csharp
namespace System.IO.Compression;

public sealed class BrotliDecompressionOptions
{
public BrotliDecompressionOptions();

public bool EnableLargeWindow { get; set; }
}
```

Add an overload to `BrotliStream`:

```csharp
namespace System.IO.Compression;

public sealed partial class BrotliStream
{
public BrotliStream(
Stream stream,
BrotliDecompressionOptions decompressionOptions,
bool leaveOpen = false);
}
```

The lower-level `BrotliDecoder` should expose equivalent functionality:

```csharp
namespace System.IO.Compression;

public partial struct BrotliDecoder
{
public BrotliDecoder(BrotliDecompressionOptions decompressionOptions);
}
```

The default value of `EnableLargeWindow` would be `false`, preserving the existing RFC 7932 behavior.

When enabled, the decoder would configure the underlying Brotli implementation equivalently to:

```c
BrotliDecoderSetParameter(
state,
BROTLI_DECODER_PARAM_LARGE_WINDOW,
BROTLI_TRUE);
```

An alternative API design could expose the maximum accepted window size instead of a Boolean:

```csharp
public sealed class BrotliDecompressionOptions
{
public int MaxWindowLog2 { get; set; } = 24;
}
```

This could allow values up to `30`.

I slightly prefer `EnableLargeWindow` because Large Window is a distinct Brotli extension and directly corresponds to `BROTLI_DECODER_PARAM_LARGE_WINDOW`, but `MaxWindowLog2` may provide better control over memory limits.

### API Usage

## API Usage

Stream-based decompression:

```csharp
using FileStream input = File.OpenRead("large-window.br");

var options = new BrotliDecompressionOptions
{
EnableLargeWindow = true
};

using var brotli = new BrotliStream(
input,
options,
leaveOpen: false);

using FileStream output = File.Create("output.bin");

brotli.CopyTo(output);
```

Without explicitly enabling Large Window, behavior remains unchanged:

```csharp
using var brotli = new BrotliStream(
input,
CompressionMode.Decompress);
```

Lower-level decoding could use the same option:

```csharp
var options = new BrotliDecompressionOptions
{
EnableLargeWindow = true
};

using var decoder = new BrotliDecoder(options);

OperationStatus status = decoder.Decompress(
source,
destination,
out int bytesConsumed,
out int bytesWritten);
```

For a PDF parser, this would allow the standardized `BrotliDecode` filter to be implemented without adding another Brotli implementation or native platform-specific dependencies:

```csharp
var options = new BrotliDecompressionOptions
{
EnableLargeWindow = true
};

using var decoded = new BrotliStream(
encodedPdfStream,
options);

decoded.CopyTo(output);
```

### Alternative Designs

_No response_

### Risks

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.