Cached failed action is used when retrying a failed build
- Dominant language
- Python
- Stars
- 140
- Forks
- 45
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 6
Description
When using remote execution, if a build fails the failure is saved in the action cache. However builds can fail for a variety of reasons (e.g. running out of resources) that are not necessarily reproducible.
Buildstream offers to retry at the command line and also offers the `--retry-failed` command line argument for this purpose. However, it will still get a cached failure from the action cache.
Here is an example log (I extracted just the things relevant to the element in question):
```
[11:08:35][--:--:--][8524a2f2][ pull:freedesktop-sdk.bst:components/fontconfig.bst] START freedesktop-sdk/components-fontconfig/8524a2f2-pull.20250611-110835.log
[11:08:35][--:--:--][8524a2f2][ pull:freedesktop-sdk.bst:components/fontconfig.bst] STATUS Pulling artifact 8524a2f2 <- https://gbm.gnome.org:11004
[11:08:35][--:--:--][8524a2f2][ pull:freedesktop-sdk.bst:components/fontconfig.bst] STATUS Pulling data for artifact 8524a2f2 <- https://gbm.gnome.org:11004
[11:08:35][--:--:--][8524a2f2][ pull:freedesktop-sdk.bst:components/fontconfig.bst] INFO Pulled artifact 8524a2f2 <- https://gbm.gnome.org:11004
[11:08:35][--:--:--][8524a2f2][ pull:freedesktop-sdk.bst:components/fontconfig.bst] INFO Discarded failed build
[11:08:35][00:00:00][8524a2f2][ pull:freedesktop-sdk.bst:components/fontconfig.bst] SKIPPED Pull
[11:08:35][--:--:--][8524a2f2][ fetch:freedesktop-sdk.bst:components/fontconfig.bst] START freedesktop-sdk/components-fontconfig/8524a2f2-fetch.20250611-110835.log
[11:08:35][--:--:--][8524a2f2][ fetch:freedesktop-sdk.bst:components/fontconfig.bst] STATUS Pulling source 031032b8 <- https://gbm.gnome.org:11004
[11:08:36][--:--:--][8524a2f2][ fetch:freedesktop-sdk.bst:components/fontconfig.bst] STATUS Pulling data for source 031032b8 <- https://gbm.gnome.org:11004
[11:08:36][--:--:--][8524a2f2][ fetch:freedesktop-sdk.bst:components/fontconfig.bst] INFO Pulled source 031032b8 <- https://gbm.gnome.org:11004
[11:08:36][00:00:00][8524a2f2][ fetch:freedesktop-sdk.bst:components/fontconfig.bst] SUCCESS freedesktop-sdk/components-fontconfig/8524a2f2-fetch.20250611-110835.log
[11:08:37][--:--:--][8524a2f2][ build:freedesktop-sdk.bst:components/fontconfig.bst] START freedesktop-sdk/components-fontconfig/8524a2f2-build.20250611-110837.log
[11:08:37][--:--:--][8524a2f2][ build:freedesktop-sdk.bst:components/fontconfig.bst] START Staging dependencies at: /
[11:09:04][00:00:26][8524a2f2][ build:freedesktop-sdk.bst:components/fontconfig.bst] SUCCESS Staging dependencies at: /
[11:09:04][--:--:--][8524a2f2][ build:freedesktop-sdk.bst:components/fontconfig.bst] START Integrating sandbox
[11:09:32][--:--:--][8524a2f2][ build:freedesktop-sdk.bst:components/fontconfig.bst] INFO Action result found in action cache
[11:09:48][00:00:44][8524a2f2][ build:freedesktop-sdk.bst:components/fontconfig.bst] SUCCESS Integrating sandbox
[11:09:48][--:--:--][8524a2f2][ build:freedesktop-sdk.bst:components/fontconfig.bst] START Staging sources
[11:09:48][00:00:00][8524a2f2][ build:freedesktop-sdk.bst:components/fontconfig.bst] SUCCESS Staging sources
[11:09:48][--:--:--][8524a2f2][ build:freedesktop-sdk.bst:components/fontconfig.bst] START Running commands
[11:09:48][--:--:--][8524a2f2][ build:freedesktop-sdk.bst:components/fontconfig.bst] INFO Action result found in action cache
[11:09:49][00:00:00][8524a2f2][ build:freedesktop-sdk.bst:components/fontconfig.bst] FAILURE Running commands
[11:09:49][--:--:--][8524a2f2][ build:freedesktop-sdk.bst:components/fontconfig.bst] START Caching artifact
[11:09:49][00:00:00][8524a2f2][ build:freedesktop-sdk.bst:components/fontconfig.bst] SUCCESS Caching artifact
[11:09:49][00:01:11][8524a2f2][ build:freedesktop-sdk.bst:components/fontconfig.bst] FAILURE Command failed
```
You can see that bst pulled a cached failure and promptly discarded it, but it then proceeded to get the failed action from the action cache and not actually retry the build.
Contributor guide
Research direction
Start by tracing the --retry-failed command-line path and the action-cache lookup used during remote execution. Reproduce the logged scenario with a failed cached action; done means retrying bypasses the cached failure and runs the build action again.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100