balderdashy / balderdashy/sails

Memory leak when lifting and lowering Sails repeatedly in automated tests

Open
#6,804 25 comments 0 reactions 0 assignees View on GitHub
bug pr welcome
Dominant language
JavaScript
Stars
22.8k
Forks
1.9k
PR merge metrics
No merged PRs in 30d

Description

**Node version**: v10.16.0
**Sails version** _(sails)_: 1.2.3
**ORM hook version** _(sails-hook-orm)_: Not using this.
**Sockets hook version** _(sails-hook-sockets)_: Not using this.
**Organics hook version** _(sails-hook-organics)_: Not using this.
**Grunt hook version** _(sails-hook-grunt)_: Not using this.
**Uploads hook version** _(sails-hook-uploads)_: Not using this.
**DB adapter & version** _(e.g. sails-mysql@5.55.5)_: Not using this.
**Skipper adapter & version** _(e.g. skipper-s3@5.55.5)_: Not using this.


I've looked through all issued related to memory leaks, but none of them describes my case.

When testing a sails application, we 'lift' and 'lower' for each test. We have an extensive test suite, and we now experience OOM errors when running tests. We use the supported constructor as mentioned here: https://sailsjs.com/documentation/concepts/programmatic-usage

I managed to make a tiny script that clearly shows the increasing memory footprint.

``` javascript
const Sails = require("sails").Sails;

(async () => {
for(let i = 0; i < 500000; i++){
await startStop();
if(i % 100 === 0) printUsage();
}
})();

async function startStop() {
const sails = new Sails();
await sails.lift({ log: { noShip: true }, session: { secret: 'x' } });
await sails.lower();
}

function printUsage() {
const used = process.memoryUsage().heapUsed / 1024 / 1024;
console.log(`Process uses ${Math.round(used * 100) / 100} MB`);
}
```

It takes awhile, but eventually it runs out of memory. Some memory is clearly being freed by the garbage collector, but some is never freed. Output from my console is:

```
ath88@lipton:~/test/sails-test$ node index.js
Process uses 23.84 MB
Process uses 29.99 MB
Process uses 34.82 MB
...
Process uses 1271.33 MB
Process uses 1286.41 MB
Process uses 1287.27 MB

<--- Last few GCs --->

[11916:0x348d020] 500144 ms: Mark-sweep 1239.6 (1418.2) -> 1239.6 (1395.2) MB, 713.7 / 0.1 ms (average mu = 0.830, current mu = 0.000) last resort GC in old space requested
[11916:0x348d020] 500840 ms: Mark-sweep 1239.6 (1395.2) -> 1239.6 (1395.2) MB, 695.9 / 0.1 ms (average mu = 0.711, current mu = 0.000) last resort GC in old space requested

<--- JS stacktrace --->

==== JS stack trace =========================================

0: ExitFrame [pc: 0xf994af5be1d]
1: StubFrame [pc: 0xf994af5a1bb]
Security context: 0x345970a9e6e9
2: set [0x345970a937b9](this=0x221dd5702859 ,0x300fe1705e19 ,0x300fe1705da9 )
3: /* anonymous */ [0x221dd57069d1] [/home/ath88/test/sails-test/node_modules/@sailshq/lodash/lib/index.js:~7645] [pc=0xf994af707d4](this=0x1e819d409ad1 v8::internal::Factory::NewFixedArrayWithMap(v8::internal::Heap::RootListIndex, int, v8::internal::PretenureFlag) [node]
10: 0x10089b1 v8::internal::HashTable::NewInternal(v8::internal::Isolate*, int, v8::internal::PretenureFlag) [node]
11: 0x1008a0e v8::internal::HashTable::New(v8::internal::Isolate*, int, v8::internal::PretenureFlag, v8::internal::MinimumCapacity) [node]
12: 0x101926f v8::internal::HashTable::EnsureCapacity(v8::internal::Handle, int, v8::internal::PretenureFlag) [node]
13: 0x102b618 v8::internal::ObjectHashTable::Put(v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, int) [node]
14: 0x102b836 v8::internal::JSWeakCollection::Set(v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, int) [node]
15: 0x1142060 v8::internal::Runtime_WeakCollectionSet(int, v8::internal::Object**, v8::internal::Isolate*) [node]
16: 0xf994af5be1d
Aborted (core dumped)
```

As far as I can see from snapshots with Chrome DevTools, `async` is getting re-required and never removed from memory.

If would be awesome to eliminate this memory leak, so we can continue our testing practices, with the supported 'programmatic usage'.

Contributor guide

Open the contributing guide

Research direction

Start by running the index.js reproduction with Node v10.16.0 and inspect Chrome DevTools heap snapshots around repeated Sails lift/lower cycles. Pay particular attention to the reported async references and the lodash stack at node_modules/@sailshq/lodash/lib/index.js:~7645; done means repeated cycles no longer show unbounded retained memory or end in an out-of-memory failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.