apache / apache/openwhisk

Reactive prewarm pool

Open
#4,725 3 comments 2 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
6.8k
Forks
1.2k
Avg merge
2d 14h
Merged PRs (30d)
2

Description

Feature suggestion: Instead of prewarm config being statically defined at deployment, make the pool behavior reactive to load. This may be more applicable to cluster managed resources (mesos/k8s/yarn) where each invoker is not restricted to local resources for launching actions.

Since reliance on prewarm containers is a key point in improving performance, we should consider ways to keep as many as possible "cold start" containers in the "prewarm" workflow.

As an example, in current implementation where a fixed number of prewarms is launched at start, and the prewarm pool is replenished each time one is taken for use:
* start with 10 prewarms
* load of 15 actions will use 10 prewarms + 5 cold containers
* 10 prewarms are replaced

Doing this once for a burst of traffic may be an anomaly, but if it is a pattern every few minutes, we can easily begin to see that we often run at a deficit of prewarms.
It would be nice to allow operators to define rules around prewarms like:
```
nodejs:10-256MB
period - 1 minute
threshold - 4
miss-count - 2
prewarm-increment - 3
```
So that if there are 2 consecutive 1 minute periods where number of prewarm "miss" (aka cold starts) for 256MB nodejs:10 activations exceed 4, then 3 additional prewarms are added to the system.

We would also need some form of "prewarm idle release" process, similar to the existing idle timeout, to reduce the number of prewarms if they get to a state of being unused.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.