bazelbuild / bazelbuild/continuous-integration

Make "publish binaries" pipeline more robust

Open
#1,075 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
302
Forks
194
Avg merge
1d 19h
Merged PRs (30d)
41

Description

Example failure:

https://buildkite.com/bazel-trusted/publish-bazel-binaries/builds/8525#7ce8bb29-4615-470e-a9aa-d20ede99e03d

This was an intermittent failure (all other builds, including a retry, succeeded) that went unnoticed until a user reported problems with Bazelisk: https://github.com/bazelbuild/continuous-integration/issues/1074

Two action items:
1. Investigate this particular flake
2. Upload as many binaries as possible, even if some tasks failed. Currently the upload step only runs if all tasks succeed.

Contributor guide

Open the contributing guide

Research direction

Start with the failed Buildkite run for the publish-bazel-binaries pipeline and compare it with the successful retry and the related Bazelisk report. Trace the pipeline's task and upload steps; done means identifying the intermittent failure and ensuring available binaries are uploaded when individual tasks fail.

Written by the indexing model from the issue text.

Assessment

Domain
build-system, ci-cd, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.