web-platform-tests / web-platform-tests/wpt
MANIFEST.json.zst files are bigger than bz2 files
Nobody has claimed this yet.
- Dominant language
- HTML
- Stars
- 6.2k
- Forks
- 3.9k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 586
Description
While looking into https://github.com/web-platform-tests/wpt/pull/33452 and trying to avoid the zstd binary as a dependency, I discovered that the MANIFEST.json.zst files are actually bigger than the bzip2 files we're uploading. In https://github.com/web-platform-tests/wpt/releases/tag/merge_pr_33467, we have:
MANIFEST.json.bz2: 2.08 MB (2180027 bytes)
MANIFEST.json.gz: 2.54 MB (2662852 bytes)
MANIFEST.json.zst: 2.09 MB (2190299 bytes)
In this case, zst is ~0.5% bigger than bz2.
bz2 and zst formats were introduced in https://github.com/web-platform-tests/wpt/pull/16083, but at the time zst was smaller:
MANIFEST.json.bz2: 1.94 MB
MANIFEST.json.gz: 2.37 MB
MANIFEST.json.zst: 1.86 MB
Since it requires an additional Python dependency we should reconsider if using zst is still really useful.
cc @gsnedders @jgraham
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file or test is named. Start by locating the release path that produces and uploads MANIFEST.json.bz2, MANIFEST.json.gz, and MANIFEST.json.zst, then compare its zstd dependency and compression choices. Done means a decided compression approach with the release artifacts and dependency requirements aligned.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- release, tooling
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100