dotnet / dotnet/project-system

Update pipeline artifact naming/build numbering for official.yml

Open
#8,431 0 comments 0 reactions 0 assignees View on GitHub
Area-Infrastructure Priority:3 Triage-Approved
Dominant language
C#
Stars
1k
Forks
415
PR merge metrics
No merged PRs in 30d

Description

### Description/Concerns
Currently, we use `$(Build.BuildNumber)` (for the primary build artifact) which is the value of that particular pipeline run and doesn't relate to our build (assemblies/packages) directly. You can [see here](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/run-number?view=azure-devops&tabs=yaml) that the default format is: `$(Date:yyyyMMdd).$(Rev:r)`. For example, you'd get a value like `20220826.7`. Also note that you can change this value by setting `name:` on the pipeline. Our previous reliance on RoslynInsertionTool required it to [use the default `Build.BuildNumber` as the artifact name](https://github.com/dotnet/roslyn-tools/blob/1f64f56b4ea9a0c119bbe840c39b935acc60e4b8/src/RoslynInsertionTool/RoslynInsertionTool/RoslynInsertionTool.VisualStudioTeamServices.cs#L174). Since we no longer use that tool, we have some freedom in what we name the artifact.

I've previously tried changing `Build.BuildNumber` to `GitBuildVersion` as part of the [mechanism in NerdBank.GitVersioning](https://github.com/dotnet/Nerdbank.GitVersioning/blob/main/doc/cloudbuild.md#automatically-match-cloud-build-numbers-to-to-your-git-version). Since this was problematic with RoslynInsertionTool, I removed that change. We can consider changing this again, but my next point below would need a custom solution *specifically* for the artifact name.

There is another problem where we cannot currently retry certain jobs in the build. Why? Because the artifacts once published [are immutable](https://developercommunity.visualstudio.com/t/publish-artifact-fails-when-retry-failed-job/774908). This means we cannot republish the artifacts with the same name in the same pipeline run. The reason we even publish the artifacts on a failed job is so that someone can debug the failure (download the artifacts and investigate). So, if we want to allow rerunning failed jobs (which saves time over rerunning the pipeline entirely), we would need a *rolling name for all artifacts*. This name needs to be accessible by all the jobs in the pipeline, or else jobs cannot access artifacts from previous jobs. For example, the artifacts from the Build job are downloaded by the Publish jobs. Some solutions [like this one](https://github.com/microsoft/azure-pipelines-tasks/issues/12854#issuecomment-645707122) may be used but needs to be tested. When retrying occurs currently, we get an error such as:
> ##[error]Artifact 20220812.10 already exists for build 6553163.

Lastly, we also use `$(Build.BuildNumber)` as part of our drop location for VS insertion files. We would want to change that to something that, again, relates to the build itself or a combination therein (so republishing the drop doesn't cause issues).

### List of Benefits
- Systems related to the build can relate more directly with the code that is being built
- Example: Relating it by CommitId could help tracking information easier when looking through history/debugging builds
- Allows for ability to retry jobs using a rolling number mechanism
- VS Insertion drops will show the code's build version number instead of pipeline run build number
- Build pipeline name/number can be updated to reflect something more appropriate with the code being built

### Side-Note
I've been stewing over this situation for a while and I didn't address it when [consolidating the pipelines](https://github.com/dotnet/project-system/pull/8420) as it has all these thorny edges to it. Multiple solutions need to be tested quite a bit since this value is used in many places.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.