apache / apache/openwhisk

fabric8 kubernetes client timeout precision

Open
#4,812 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
6.8k
Forks
1.2k
Avg merge
2d 14h
Merged PRs (30d)
2

Description

## Environment details:

* Kubernetes

## Steps to reproduce the issue:

1. StandaloneOpenwhisk openwhisk with invalid docker image specified for a prewarm kind
2. set the kubernetes run timeout to 60s
3. start StandaloneOpenwhisk

e.g.
```
whisk {
kubernetes {
timeouts {
run = 60 seconds
```

## Provide the expected results and outputs:

Timeout error occurs for prewarm startup after the configured timeout(60s) with some reasonable overhead for execution etc. e.g. I expect the actual timeout to occur with 10s of the configured timeout.

## Provide the actual results and outputs:

Timeout for kubernetes run is much longer than expected, like 30+seconds longer, but this difference changes with the scale of the configured timeout.

## Additional information you deem important:
The actual timeout seems _relative_ to the configured timeout, but in no cases was I able to make it behave _close_ to the configured timeout.

I did take a look at the fabric8 client, and saw several past issues related to timeout handling. Looking at the code, I wasn't able to quickly determine what the problem is, but there are some Thread.sleep() calls buried within that I am concerned about.

A couple of options I've considered:
* don't use fabric8, just use the "official" kubernetes java client - any reason not to do this?
* replace the fabric8 timeout handling with a polling approach + akka "after" pattern
* fix fabric8 - as mentioned I wasn't able to see an easy route to this (I could not get the project to load into intellij, so didn't look much further...). In addition to the Thread.sleep usage, I also saw and interesting approach to timeout of splitting the timeout into operations that each run every 1/10 of the timeout period, as opposed to consistently polling every :
https://github.com/fabric8io/kubernetes-client/blob/db35f4c7c6dadc8e2740f234ee68dd23d4bb39ff/kubernetes-client/src/main/java/io/fabric8/kubernetes/client/dsl/base/HasMetadataOperation.java#L205

For simplicity, I'm wondering if there is any reason not to use the "plain" kubernetes java client?

@dgrove-oss Do you have any thoughts on this?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.