Tippecanoe holds forever above 5 million points in geojson file
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 3.1k
- Forks
- 430
- PR merge metrics
- No merged PRs in 30d
Description
I have a gzipped file `positions.json.gz` with one geojson object representing one point per line:
```json
{"type":"Feature","geometry":{"type":"Point","coordinates":[-88.55666,30.10603]},"properties":{"speed":0}}
{"type":"Feature","geometry":{"type":"Point","coordinates":[-88.55661,30.10625]},"properties":{"speed":0}}
{"type":"Feature","geometry":{"type":"Point","coordinates":[-88.55695,30.106083]},"properties":{"speed":0}}
```
I run tippecanoe on Ubuntu 16.04 in parallel mode with the following command:
```shell
tippecanoe -o positions.mbtiles -P -z 22 -B 15 -r 1.25 --maximum-tile-bytes=5000000 positions.json.gz --force
```
This works great when there are fewer than approximatively 5 million rows in the file. As the count of rows increases, I noticed that tippecanoe takes more time to really start "processing" the points across multiple CPU cores. This seems normal as I'm assuming that tippecanoe simply loads the rows in memory before starting the parallel computation.
However, above 5 million rows, this "loading" step seems to hold forever and the multi-processing computation never actually starts. I've waited for several hours with 10 million rows with nothing happening.
Is this a bug or am I doing anything wrong?
*note 1: if I remove `-P` from the command to let tippecanoe run on a single core, the behaviour doesn't change: it will hold forever above 5 million points as well*
*note 2: if I don't pass any of the `-z`, `-B`, `-r` or `--maximum-tile-bytes` parameters then behaviour is still the same again*
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported hang with the supplied tippecanoe command and positions.json.gz, comparing inputs below and above five million rows and with and without -P. No source file or test is named in the issue; done means identifying why processing does not begin at larger inputs and verifying a fix against the reported cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- cli, data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100