digidem / digidem/styled-map-package

Packages written with dedupe can't be read when the caller constructs the ZipReader

Open
#125 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
4
Forks
1
Avg merge
11h 36m
Merged PRs (30d)
17

Description

Writing with new Writer(style, { dedupe: true }) produces an archive that @gmaclennan/zip-reader rejects by default:

Error: Duplicate local file header offset detected (possible ZIP bomb)

Dedupe points several central directory entries at one local file header, which is also the ZIP bomb technique zip-reader guards against (dist/index.js, in the async iterator).

Reader handles this when it builds the reader itself — lib/reader.js:209 passes skipUniqueEntryCheck: true — so new Reader('/path/to/file.smp') works. But the documented browser path is to construct the ZipReader yourself:

const reader = new Reader(await ZipReader.from(new BlobSource(file)))

That gets zip-reader's defaults, so every getResource() for a deduplicated tile throws and nothing renders. It only shows up with tilesets that repeat tiles (ocean tiles, single-colour tiles); packages whose tiles are all distinct read fine, which makes it easy to miss.

Found in a browser SMP viewer: exports of a 9.6MB countries-png.mbtiles opened as an empty map, while an OpenFreeMap package read fine. Passing skipUniqueEntryCheck: true when constructing the ZipReader fixes it. This also affects packages written by map-downloader, which writes with dedupe: true.

lib/validator.js:127 builds ZipReader.from(fileSource) without the option too, so validating a deduplicated package should fail the same way (not verified).

Possible fixes, in rough order of preference:

  1. Have Reader apply skipUniqueEntryCheck to a ZipReader it is handed as well — or expose an option — so callers can't get this wrong. The Reader already caps resource size, so the bomb protection it replaces isn't doing much here.
  2. Document it prominently for the browser path, since the failure mode is an empty map rather than an obvious error.
  3. Reconsider dedupe writing archives the ecosystem's own reader rejects by default.

Versions: styled-map-package-api 6.0.1, @gmaclennan/zip-reader 1.0.0.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at lib/reader.js:209 and compare how Reader handles an internally created ZipReader with one supplied by the caller. Check lib/validator.js:127 and the documented browser construction path, then reproduce with a deduplicated package. Done means deduplicated resources can be read through the supplied ZipReader without breaking the existing resource-size protection.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
api
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.