HPA doesn't scale down to minimum size when no data is available from external metrics API server

When I try to fetch metrics from external metrics endpoint (such as stackdriver), stackdriver will stop sending metrics if data is not available. For example, the following command will confirm if the adapter is receiving any data from stackdriver or not.

```
kubectl get --raw "/apis/external.metrics.k8s.io/v1beta1/namespaces/default/loadbalancing.googleapis.com|l3|internal|ingress_packets_count" | jq
```

If the data is not available, the value reported by HPA will go into an unknown state and no scaling activity will happen.

Is it possible to build logic into the adapter if data is not available ore reported, then the HPA automatically scales down the metric to minimum size ?