google / google/brotli

Custom dictionaries below level 5

Open
#1,148 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
14.9k
Forks
1.4k
Avg merge
4d 20h
Merged PRs (30d)
9

Description

Custom dictionaries can be attached during compression and decompression using C APIs like `BrotliEncoderAttachPreparedDictionary`. However, [it appears they aren't used below brotli level 5](https://blog.cloudflare.com/brotli-compression-using-a-reduced-dictionary). This causes a kind of silent failure where the user doesn't observe the lack of improvement.

There's a few solutions to this issue
1. The attach and/or prepare methods could document the behavior.
2. These and `BrotliEncoderPrepareDictionary` could fail when an incompatible level is applied.
3. The dictionary format could include the minimum brotli level for compatibility & API-simplification reasons.
4. Decide this is a bug and implement the feature for lower levels (not backwards compatible)

Here's an example in case the issue isn't clear
```sh
brotli -0 -D dictionary.lz -o dict
brotli -0 no-dict
diff dict no-dict # would use <() but this doesn't work either
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.