apache / apache/openwhisk

Performance: the latency and total run time increase when invoker_user_memory increase?

Open
#4,414 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
6.8k
Forks
1.2k
Avg merge
2d 14h
Merged PRs (30d)
2

Description

## Environment details:

* local deployment, native ubuntu
* ubuntu16,
* CPU:Xeon(R) CPU E5-2630 v4 * 40
* memory:252G
* all the ow component is deployed in a server with one controller and one invoker.

## Steps to reproduce the issue:
We first define a simple Node.js action and save it as sleepy.js in the local file system. This action is anything but sophisticated: it just sleeps for a number of milliseconds as specified in its argument and echoes back the argument.

```
function main(args) {
return new Promise(function(resolve, reject) {
setTimeout(function() {
resolve({ timeout: args.timeout });
}, args.timeout);
})
}
```

We use this simple action only to stress OW, and deploy invoker with `invoker_user_memory` increase from 2048m to 10240M, step by 2048m.

Then we test OW with `loadtest`(the detail script [here](https://github.com/axiqia/lean-openwhisk-performance/blob/master/repeats.sh)), the test iterates over the specified test configurations.

Each test is replicated 15 times as specified by the repeats parameter to gain statistical significance of the results([detailed](https://github.com/axiqia/lean-openwhisk-performance/blob/master/alllog)).

## Provide the actual results and outputs:

I increase `invoker_user_memory` for invoker, and it should have more capacity to run more container a time, but our experiment bellow shows that the latency and total run time increase instead of declining.

![latency](https://github.com/axiqia/lean-openwhisk-performance/raw/master/latency.png)
![totaltime](https://github.com/axiqia/lean-openwhisk-performance/raw/master/totaltime.png)

The detailed data and figure is saved as [xlsx](https://github.com/axiqia/lean-openwhisk-performance/blob/master/plot.xlsx)
And some time, it return 429 error.
## Relevant issues
- https://lists.apache.org/thread.html/dfccf972bc1419fe48dbc23119441108c45f85d53625fd6f8fc04fcb@%3Cdev.openwhisk.apache.org%3E
- https://github.com/apache/incubator-openwhisk/issues/2515

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.