knative / knative/serving

The conformance test ContainerExitingMsg is very flaky and currently dependant on a custom progressDeadline

Open
#13,465 2 comments 0 reactions 0 assignees View on GitHub
area/API area/test-and-release kind/bug triage/accepted
Dominant language
Go
Stars
6.1k
Forks
1.2k
Avg merge
2d 7h
Merged PRs (30d)
2

Description

The conformance test [ContainerExitingMsg](https://github.com/knative/serving/blob/aebc16c5671faf383e25c99af95530e3228e7e28/test/conformance/api/v1/errorcondition_test.go#L120 ) seems to be very flaky. From what I understand the test wants to hit [this code here](https://github.com/knative/serving/blob/aebc16c5671faf383e25c99af95530e3228e7e28/pkg/reconciler/revision/reconcile_resources.go#L83) to set the revision to `False` if the pod crashes repeatedly. However, this is almost never reached since the deployment is reconciled every time (https://github.com/knative/serving/issues/13204) and may run into a race condition while trying to update it preventing it from ever reaching that code.

## In what area(s)?
/area API
/area test-and-release

## What version of Knative?

> 1.6+

## Expected Behavior

Conformance test `ContainerExitingMsg` passes.

## Actual Behavior

Conformance test `ContainerExitingMsg` is not passed with knative defaults.
https://github.com/knative/serving/actions/runs/3444537426/jobs/5747350496
`PASS test/conformance/api/v1.TestContainerExitingMsg (121.05s)` this is using customized values in the knative pipeline and passes only because the custom progressDeadline is set to 120s (https://github.com/knative/serving/blob/main/test/config/ytt/core/overlay-config-deployment.yaml). The actual default ProgressDealine is set to 10m (the same as the default timeout of the conformance tests) which would prevent this test from passing most of the time.

## Steps to Reproduce the Problem
Run the conformance tests with a 10m timeout on a knative installation with a default progressDeadline of 10m.

Contributor guide

Open the contributing guide

Research direction

Start with test/conformance/api/v1/errorcondition_test.go at ContainerExitingMsg and reproduce it with the default 10m progressDeadline. Read pkg/reconciler/revision/reconcile_resources.go and issue #13204 to understand the reported reconciliation race, then compare test/config/ytt/core/overlay-config-deployment.yaml. Done means the conformance test passes with Knative defaults rather than custom pipeline values.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
api, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.