Golden tests failed to run on the 3.23 beta release
- Dominant language
- Dart
- Stars
- 179k
- Forks
- 31.1k
- PR merge metrics
- PR metrics pending
Description
Previously [work was done](https://github.com/flutter/engine/tree/main/testing/skia_gold_client#release-testing) to make sure the golden tests run on the release branches. It was verified to work in a mock run at https://github.com/flutter/flutter/issues/146078. When the [3.23 beta branch was created golden images were not reaped](https://github.com/flutter/engine/pull/53148) like it did in the mock test.
This is likely a difference in the environment between the test and the real release workflow. The code that added the ability for cocoon to run the skia gold on release branches was added in https://github.com/flutter/cocoon/pull/3659 and appears intact still.
The PR that disabled the golden image tests for the 3.23 beta branch was https://github.com/flutter/engine/pull/53163. It appears that the release team ran into issues beyond just the failure of cocoon to reap the golden images too since there was a need to disable the tests. I believe that is their GOLDCTL env var issue which should be resolved by adding skia gold as a dependency in the builder.
We need to identify the difference between the mock run and the real workflow to understand why cocoon did not reap the golden images.
cc @matanlurey
Contributor guide
Assessment
This issue has not been assessed yet.