mapbox / mapbox/tile-reduce

optionally hit worker even if some sources didn't hold data for the tile

Open
#75 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
JavaScript
Stars
185
Forks
31
PR merge metrics
No merged PRs in 30d

Description

Per [this code](https://github.com/mapbox/tile-reduce/blob/5aaf09f9b196f4d5fe0fda9f76884e75b7c37db8/src/worker.js#L42-L45):

``` js
for (var i = 0; i < results.length; i++) {
data[sources[i].name] = results[i];
if (!results[i]) return process.send({reduce: true});
}
```

the worker bails out and returns a reduce event if any source doesn't have data for the requested tile. This is usually great, but in some cases where you want to compare disparate data sources and are relying on reduce events to send back information about how much data each source does or doesn't exist in a tile, you end up losing information.

For example, if I want to find the length of roads in San Francisco that are matched by GPS datapoints. I would like to keep a tally of the total length of road in the bbox, as well as how much is matchable by GPS points. Right now, if there is no GPS data in the tile, we bail out, so I'm missing some of the total length information.

To maintain compatibility and provide optimization for the usual cases where you _want_ this bail-out behavior, I'm proposing we add a tile-reduce option for this, maybe `requireAllSources: false` (defaulted true).

cc @morganherlocker @aaronlidman @mourner

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/worker.js at the loop in lines 42-45, then trace how tile-reduce options reach the worker and how reduce events are emitted. Add an option that preserves the current default bail-out behavior while allowing processing when a source has no data, and verify that reduce events retain the available source information.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
data
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
40/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.