kubernetes / kubernetes/kubectl

kubectl wait sometime doesn't resolve ready condition

Open
#1,414 20 comments 3 reactions 0 assignees View on GitHub
kind/bug lifecycle/rotten needs-triage
Dominant language
Go
Stars
3.3k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

**What happened**:
I have a CRD with conditions.
The status of the conditions:

```
status:
...
conditions:
- lastTransitionTime: "2023-04-14T20:38:28Z"
status: "True"
type: IngressReady
- lastTransitionTime: "2023-04-14T20:38:28Z"
status: "True"
type: PredictorReady
- lastTransitionTime: "2023-04-14T20:38:28Z"
status: "True"
type: Ready
...
```

When I wait for the condition, I expect it to return successfully, but instead, I get timed-out error.

```
k wait isvc test1 --for=condition=ready=true --timeout=5s
error: timed out waiting for the condition on inferenceservices/test1
```

**What you expected to happen**:

```
k wait isvc test2 --for=condition=ready=true --timeout=5s
inferenceservice.serving.kserve.io/test2 condition met
```

Both services report the same status, but kubectl only works for one of them.
I got the kubectl output with -v=10 debug level, but no issues there, both for the object that times out and for other object conditions response look the same:

OK:
```
"conditions":[{"lastTransitionTime":"2023-04-14T19:36:40Z","status":"True","type":"IngressReady"},{"lastTransitionTime":"2023-04-14T19:36:40Z","status":"True","type":"PredictorReady"},{"lastTransitionTime":"2023-04-14T19:36:40Z","status":"True","type":"Ready"}]
```

Failed:
```
"conditions":[{"lastTransitionTime":"2023-04-14T21:24:21Z","status":"True","type":"IngressReady"},{"lastTransitionTime":"2023-04-14T21:24:21Z","status":"True","type":"PredictorReady"},{"lastTransitionTime":"2023-04-14T21:24:21Z","status":"True","type":"Ready"},{"lastTransitionTime":"2023-04-08T02:16:46Z","severity":"Info","status":"True","type":"TransformerReady"}]
```

The failing one also contains severity, but not sure if it's related
field comes from https://github.com/knative/pkg/blob/main/apis/condition_types.go#L67-L70

**How to reproduce it (as minimally and precisely as possible)**:

**Anything else we need to know?**:
This is KServe CRDs, but not sure if it's relevant

**Environment**:
- Kubernetes client and server versions (use `kubectl version`):
```
k version --short
Flag --short has been deprecated, and will be removed in the future. The --short output will become the default.
Client Version: v1.25.4
Kustomize Version: v4.5.7
Server Version: v1.25.7-gke.1000
```
- Cloud provider or hardware configuration: GKE
- OS (e.g: `cat /etc/os-release`): COS

Contributor guide

Open the contributing guide

Research direction

Start by reproducing `kubectl wait` with the reported KServe objects, Kubernetes versions, and the two condition-list responses in the issue. Compare how the `Ready=True` condition is evaluated when the extra `TransformerReady` condition and `severity` field are present; done means equivalent ready objects consistently satisfy `--for=condition=ready=true`, with a focused regression test if the relevant wait entry point is located.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.