Memory allocation issue
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 185
- Forks
- 31
- PR merge metrics
- No merged PRs in 30d
Description
Here's a tile-reduce process that's run for the planet. I'm expecting the script to release memory after processing a tile. Instead process keeps taking up more memory space without freeing up memory for the processed tiles and fails at a point due to memory allocation issue.
Time Spent| Memory
-----------|---------
Before beginning the process|
10s|
30s|
After 1 minute|
### Map script
```
'use strict';
module.exports = function(tileLayers, tile, writeData, done) {
done();
};
```
### Reduce script
```
'use strict';
var tileReduce = require('@mapbox/tile-reduce');
tileReduce({
zoom: 12,
map: path.join(__dirname, 'index.js'),
sources: [{
name: 'osm',
mbtiles: 'latest.planet.mbtiles',
}
]
})
```
I'd need help to resolve this issue.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the tile-reduce run using the shown map script, reduce script, and latest.planet.mbtiles source, then inspect memory growth across tiles. Read the @mapbox/tile-reduce entry points involved in map and reduce processing and determine whether memory is retained after done() completes. Done means identifying a reproducible cause and confirming that memory no longer grows until allocation fails.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100