The conformance test ContainerExitingMsg is very flaky and currently dependant on a custom progressDeadline
- Dominant language
- Go
- Stars
- 6.1k
- Forks
- 1.2k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 2
Description
The conformance test [ContainerExitingMsg](https://github.com/knative/serving/blob/aebc16c5671faf383e25c99af95530e3228e7e28/test/conformance/api/v1/errorcondition_test.go#L120 ) seems to be very flaky. From what I understand the test wants to hit [this code here](https://github.com/knative/serving/blob/aebc16c5671faf383e25c99af95530e3228e7e28/pkg/reconciler/revision/reconcile_resources.go#L83) to set the revision to `False` if the pod crashes repeatedly. However, this is almost never reached since the deployment is reconciled every time (https://github.com/knative/serving/issues/13204) and may run into a race condition while trying to update it preventing it from ever reaching that code.
## In what area(s)?
/area API
/area test-and-release
## What version of Knative?
> 1.6+
## Expected Behavior
Conformance test `ContainerExitingMsg` passes.
## Actual Behavior
Conformance test `ContainerExitingMsg` is not passed with knative defaults.
https://github.com/knative/serving/actions/runs/3444537426/jobs/5747350496
`PASS test/conformance/api/v1.TestContainerExitingMsg (121.05s)` this is using customized values in the knative pipeline and passes only because the custom progressDeadline is set to 120s (https://github.com/knative/serving/blob/main/test/config/ytt/core/overlay-config-deployment.yaml). The actual default ProgressDealine is set to 10m (the same as the default timeout of the conformance tests) which would prevent this test from passing most of the time.
## Steps to Reproduce the Problem
Run the conformance tests with a 10m timeout on a knative installation with a default progressDeadline of 10m.
Contributor guide
Research direction
Start with test/conformance/api/v1/errorcondition_test.go at ContainerExitingMsg and reproduce it with the default 10m progressDeadline. Read pkg/reconciler/revision/reconcile_resources.go and issue #13204 to understand the reported reconciliation race, then compare test/config/ytt/core/overlay-config-deployment.yaml. Done means the conformance test passes with Knative defaults rather than custom pipeline values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- api, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100