mapbox / mapbox/togeojson

JavaScript heap out of memory

Open
#141 3 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
1.3k
Forks
322
PR merge metrics
No merged PRs in 30d

Description

I'm trying to convert a KML file that is 107 mb to geojson using the command line. It's a geospatial dataset from DATA.GOV about airports in the US. I installed togeojson with npm install -g @mapbox/togeojson . I'm on Mac OS High Sierra npm 5.7.1 node v8.9.4. Does anyone know how to fix the following error I'm getting Or is the KML file too big?

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node::Abort() [/usr/local/bin/node]
2: node::FatalException(v8::Isolate*, v8::Localv8::Value, v8::Localv8::Message) [/usr/local/bin/node]
3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/usr/local/bin/node]
4: v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/usr/local/bin/node]
5: v8::internal::Runtime_AllocateInTargetSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
6: 0x4994fb8463d
7: 0x4994fd364e7
8: 0x4994fd2301f
Abort trap: 6

And the output file gives the errors too

<--- Last few GCs --->

[94669:0x103001800] 24468 ms: Mark-sweep 1405.0 (1457.3) -> 1405.0 (1434.8) MB, 1996.4 / 0.0 ms (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 1996 ms) last resort GC in old space requested
[94669:0x103001800] 26579 ms: Mark-sweep 1405.0 (1434.8) -> 1405.0 (1434.8) MB, 2110.8 / 0.0 ms last resort GC in old space requested

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x22c0ad225ee1
1: createElementNS [/usr/local/lib/node_modules/@mapbox/togeojson/node_modules/xmldom/dom.js:673] [pc=0x4994fd364e7](this=0x22c02f085481 ,namespaceURI=0x22c02f0853d9 <String[30]: http://www.opengis.net/kml/2.2>,qualifiedName=0x22c0cd45ff21 <String[10]: SchemaData>)
2: appendElement [/usr/local/lib/node_modules/@mapbox/togeojson/node_modules/xmldom/sax.js:
...

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the command-line conversion with the reported 107 MB KML file and Node.js version. Inspect the parser locations shown in node_modules/xmldom/dom.js and node_modules/xmldom/sax.js, then determine whether the failure is reproducible and what supported handling or limitation should be documented. Done means the memory failure has a verified resolution or a clearly documented limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.