mapbox / mapbox/vector-tile-spec

Inefficient encoding of key/values in Layer

Open
#44 1 comment 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

3.0 Specification
Dominant language
No language data
Stars
1k
Forks
213
PR merge metrics
No merged PRs in 30d

Description

They keys and values use _non-packed_ repeated fields which is inefficient compared to _packed_ repeated fields. In addition the values are each in its own message which needs extra space for the length header. If the values are small integers that could otherwise be encoded in a single byte, this can easily mean they need six times as much space as would be needed otherwise (Value message length + Value message wire type + Value message field number + Layer message wire type + Layer message field number + the value itself).

Unfortunately this can't be changed easily, because the values can be of different types. One option would be to have a `repeated string string_value [packed=true]`, `repeated float float_value [packed=true]`, etc. in the Layer message. But this would mean reordering the keys or having multiple sets of keys for the different types.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review the vector tile specification's Layer and Value encoding definitions first. Compare the proposed packed per-type fields with the current mixed-type representation, including key/value ordering and compatibility; done requires an agreed encoding design rather than a straightforward edit.

Written by the indexing model from the issue text.

Assessment

Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.