Proposal: scale new functions down to zero which have never been invoked

Expected Behaviour

When deploying a function for the first time with com.openfaas.scale.zero: true and faas-idler's -dry-run is false, I expect my function to be idled, ie scaled down to zero after deployment but it isn't.

Current Behaviour

Currently, the function replicas will be set upon deployment to com.openfaas.scale.min which defaults to 1.

Despite the fact that com.openfaas.scale.zero: true, the faas-idler is not in dry-run mode and my function is not used, its replica stays to its minimum and my function is not idled.

The faas-idler kicks in only when the metric is present in prometheus, the function is ignored otherwise. When a function is deployed for the first time, its metrics are not yet in prometheus, they are collected only after the function is used for the first time.

Possible Solutions

1. consider 0 for functions labeled with zero scaling but without any prometheus metrics
2. using an Init Container in the function, register the 0 metric to prometheus if it isn't set yet
3. in the gateway, register the 0 metric to prometheus for deployed functions which have no metric yet

It looks to me that solution 3 is the best, less moving parts.

Note from @alexellis :

The 0 metric would be collected in the deployment handler, that or in a similar way to the gateway_service_count which is polled periodically

Optionally, another consideration is I think that however it is implemented, one should not idle a non healthy function. If the function is not healthy, doesn't start for some reason and is idled by the faas-idler, one might never know that the function is broken until the function is first used.

Context

For the context, I would like to use OpenFAAS to deploy GPU deep learning models to multiple demo environments. Since they are very heavy and costly to run, I would like them to be only started when actually used.

Your Environment

• FaaS-CLI version ( Full output from: faas-cli version ):

  ___                   _____           ____
 / _ \ _ __   ___ _ __ |  ___|_ _  __ _/ ___|
| | | | '_ \ / _ \ '_ \| |_ / _` |/ _` \___ \
| |_| | |_) |  __/ | | |  _| (_| | (_| |___) |
 \___/| .__/ \___|_| |_|_|  \__,_|\__,_|____/
      |_|

CLI:
 commit:  b24c5763d9b61e0c04018a722f8f2f765498f18a
 version: 0.7.8

Gateway
 uri:     http://127.0.0.1:8080
 version: 0.9.10
 sha:     b4c12f824bcea6b3038f5c878001f72e6a57de1e
 commit:  Make use of cache in scaling


Provider
 name:          faas-netes
 orchestration: kubernetes
 version:       0.6.3 
 sha:           62766ad0c4b2ce713df26172faa51f56b1a955ce

• Docker version docker version (e.g. Docker 17.0.05 ):

Client:
 Version:           18.06.1-ce
 API version:       1.38
 Go version:        go1.10.3
 Git commit:        e68fc7a
 Built:             Tue Aug 21 17:21:31 2018
 OS/Arch:           darwin/amd64
 Experimental:      false

Server:
 Engine:
  Version:          18.06.1-ce
  API version:      1.38 (minimum version 1.12)
  Go version:       go1.10.3
  Git commit:       e68fc7a
  Built:            Tue Aug 21 17:29:02 2018
  OS/Arch:          linux/amd64
  Experimental:     true
 Kubernetes:
  Version:          v1.10.3
  StackAPI:         v1beta2

• Are you using Docker Swarm or Kubernetes (FaaS-netes)?
  Kubernetes

• Operating System and version (e.g. Linux, Windows, MacOS):
  OSX

• Link to your project or a code example to reproduce issue:
  Verbatim java8 sample

Start by tracing the faas-idler behavior for functions without Prometheus metrics and the gateway deployment handler or gateway_service_count polling mentioned in the issue. Define how first deployments should expose a zero metric, including the non-healthy-function concern, then verify that an unused zero-scaled function reaches zero replicas without dry-run mode.