kubernetes-sigs / kubernetes-sigs/cluster-api-operator

[Failing Test] cmd/plugin `TestPreloadCommand` times out after 10m0s on OCI-override subtests

Open
#1,107 3 comments 0 reactions 0 assignees View on GitHub
area/testing kind/failing-test needs-triage
Dominant language
Go
Stars
286
Forks
131
Avg merge
2d 15h
Merged PRs (30d)
3

Description

### Which jobs are failing?

periodic-cluster-api-operator-test-main
https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/periodic-cluster-api-operator-test-main/2083657800268910592
https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/periodic-cluster-api-operator-test-main/2084020193507414016
https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/periodic-cluster-api-operator-test-main/2084382586594922496

### Which tests are failing?

sigs.k8s.io/cluster-api-operator/cmd/plugin/cmd TestPreloadCommand
sigs.k8s.io/cluster-api-operator/cmd/plugin/cmd TestPreloadCommand/multiple_providers_with_OCI_override

### Since when has it been failing?

Aug 1 2026, and on every daily run since (Aug 1, Aug 2, Aug 3 all red as of filing).
Last known-green run was Jul 31 2026: on the exact same commit `46918b6`
(https://github.com/kubernetes-sigs/cluster-api-operator/tree/46918b657109212410b71c7e0401f63b0198406d, PR #1093) as the first failing run on Aug 1.

### Testgrid link

https://storage.googleapis.com/k8s-triage/index.html?job=periodic-cluster-api-operator-test-main&text=TestPreloadCommand

### Reason for failure (if possible)

```
=== Failed
=== FAIL: cmd/plugin/cmd (0.00s)
Creating new test environment
panic: test timed out after 10m0s
running tests:
TestPreloadCommand (9m53s)
TestPreloadCommand/multiple_providers_with_OCI_override (4m27s)
...

=== FAIL: cmd/plugin/cmd TestPreloadCommand/multiple_providers_with_OCI_override (unknown)
Added file: core-cluster-api-v1.10.0-beta.0-components.yaml
Added file: core-cluster-api-v1.10.0-beta.0-metadata.yaml
Added file: infrastructure-docker-v1.10.0-beta.0-components.yaml
Added file: infrastructure-docker-v1.10.0-beta.0-metadata.yaml
Packaged manifests
```

Test hangs right after logging "Packaged manifests" (the OCI-override preload/package step), then Go's test binary is killed by the default 10-minute test timeout. The goroutine dump shows the process parked waiting on a client-go informer to sync, this looks like a hang/deadlock somewhere in the OCI-override packaging path, not an application panic.

### Anything else we need to know?

- Identical failure signature and identical two subtests fail on all 3 observed runs.
- Each occurrence burns a full 10-minute CI timeout slot.

### Label(s) to be applied

/kind failing-test
/area testing

Contributor guide

Open the contributing guide

Research direction

Start with cmd/plugin/cmd TestPreloadCommand, especially the multiple_providers_with_OCI_override subtest, and reproduce the timeout using the linked periodic job logs. Inspect the goroutine dump and the OCI-override preload/package step after "Packaged manifests". Done means the subtest completes without waiting for informer sync and the full TestPreloadCommand passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.