caolan / caolan/highland

Serious Performance Issue With Pipeline

Open
#270 2 comments 0 reactions 0 assignees View on GitHub
performance
Dominant language
JavaScript
Stars
3.4k
Forks
145
PR merge metrics
No merged PRs in 30d

Description

I've been banging my head against this for the past few days after we got some pretty bad performance results following a recent load test. At first I thought it was a memory leak in our code but on further investigation I think the increase in memory usage was because `pipeline` was causing the CPU to max out so much that the garbage collector was fighting for CPU cycles. A stripped down test case showed the CPU hitting 99-100% for some pretty trivial stream processing with the memory staying constant.

Here's the test case:

``` javascript
var _ = require('highland');

var bigArray = []
for(var i = 0; i < 100000; i++) {
bigArray[i] = i;
}
var s = _(bigArray);

function addOne(a) {
return a + 1;
}

function bigAdd(a) {
for(var i = 0; i < 500; i++) {
a++;
}
return a;
}

function bigSubtract(a) {
for(var i = 0; i < 100; i++) {
a--;
}
return a;
}

console.time('timer');
s.through(_.pipeline(_.map(addOne), _.map(bigAdd), _.map(bigSubtract))).toArray(function () {
console.timeEnd('timer');
});
```

![pipeline_cpu](https://cloud.githubusercontent.com/assets/8573472/7168957/0b41312c-e3b9-11e4-901f-867d2af55f4b.png)

This takes nearly 40 seconds to run my machine whereas this only takes 142ms:

``` javascript
console.time('timer');
s.through(_.map(addOne)).through(_.map(bigAdd)).through(_.map(bigSubtract)).toArray(function () {
console.timeEnd('timer');
});
```

If I increase the size of `bigArray` to `1000000` then the first version takes so long that I have to kill it (compared to 1.3 seconds for the other version).

The problem seems to in the `wrapper` part of the `pipeline` function:

``` javascript
var wrapper = _(function (push, next) {
end.pull(function (err, x) {
if (err) {
wrapper._send(err);
next();
}
else if (x === nil) {
wrapper._send(null, nil);
}
else {
wrapper._send(null, x);
next();
}
});
});
wrapper.write = function (x) {
start.write(x);
};
```

The thing that's stumping me is that both `pull` and `send` really don't seem to do very much so I'm not sure how they can be leading to such a massive jump in CPU use. I want to take a stab at fixing this but I'll need a couple of pointers to get me started.

Contributor guide

Open the contributing guide

Research direction

Start by running the two JavaScript reproductions in the issue and comparing their timings and CPU usage. Then inspect the shown wrapper in pipeline, tracing how pull, send, write, and next interact; done means pipeline preserves the chained transforms' results without the reported CPU and runtime spike.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.