CI: recurring flaky failures traced to a dotnet pack file-lock race and un-retried dependency downloads

## Summary

CI jobs fail often enough that re-running is a routine part of merging, and the failures
almost always pass on the second attempt. This issue records an analysis of *why*, because
the causes turn out to be a small number of fixable infrastructure problems rather than
flaky tests.

**Headline: two mechanisms account for 7 of the 11 classifiable failures. Neither is a test
being flaky — one is a build-parallelism file lock, the other is un-retried transient HTTP.**

## Method

Looking only at runs still marked `failure` is misleading: it structurally excludes every
flake that was re-run to green, which is exactly the population of interest. Instead, over
the last 120 `build.yml` runs, the runs with `run_attempt > 1` were enumerated and attempt 1
of each inspected.

- 19 runs were re-run
- 5 of those were `action_required` (fork-PR approval, not failures), 1 was cancelled
- **13 genuine failure -> re-run events, ~11% of runs**

## Breakdown

| Cause | Hits | Platforms |
|---|---|---|
| `Sentry.aar` file-lock race during `dotnet pack` | 4 | macos, linux-x64, win-x64, win-arm64 |
| Un-retried transient HTTP on a dependency fetch | 3 | win-x64 x2, MSBuild |
| Integration test `builds without warnings...` race | 2 | win-x64, MSBuild |
| Unclassified | 2 | linux-arm64, linux-musl-x64 |

---

### 1. `Sentry.aar` file-lock race during `dotnet pack` (4 hits, all platforms)

```
XARLP7024: The file is locked by: ".NET Host (3944)".
System.IO.IOException: The process cannot access the file
'src/Sentry/bin/Release/net10.0-android36.0/Sentry.aar'
because it is being used by another process.
[src/Sentry.Maui/Sentry.Maui.csproj::TargetFramework=net10.0-android36.0]
```

Raised by `ResolveLibraryProjectImports` (`Xamarin.Android.EmbeddedResource.targets(40,5)`)
during the `Create NuGet Packages` step:

https://github.com/getsentry/sentry-dotnet/blob/main/.github/workflows/build.yml#L263-L264

```yaml
- name: Create NuGet Packages
run: dotnet pack ${{ matrix.slnf }} -c Release --no-build --nologo
```

Several MSBuild nodes pack in parallel, each opens the same `Sentry.aar` to extract its
library-project imports, and one holds it without sharing, so the others fail. It is a
straight race — nothing about the diff under test matters, which is why a re-run always
clears it.

This hit **all four platforms**, so it is not a Windows-specific problem; `win-x64` just
attracts the blame because it is the job people look at.

Occurrences:
- .NET (macos) 2026-08-19 — https://github.com/getsentry/sentry-dotnet/actions/runs/32242505927/job/96036107477
- .NET (linux-x64) 2026-08-18 — https://github.com/getsentry/sentry-dotnet/actions/runs/32091470517/job/95574420783
- .NET (win-arm64) 2026-08-14 — https://github.com/getsentry/sentry-dotnet/actions/runs/31763593363/job/94654919102
- .NET (win-x64) 2026-08-27 — https://github.com/getsentry/sentry-dotnet/actions/runs/33028776243/job/98376330159

**Proposed fix:** serialise the pack step only — `-m:1` (or `-p:BuildInParallel=false`).
Pack is short next to the build, so the cost should be small, but it is worth measuring.
Worth also understanding why `ResolveLibraryProjectImports` runs at all under `--no-build`.

---

### 2. Un-retried transient HTTP on a dependency fetch (3 hits)

Three separate single `GET`s, each of which aborts the entire job when it fails.

**Maven Central `403` on the sentry-android-ndk POM — twice:**
- 8.52.0, .NET (win-x64) 2026-08-14 — https://github.com/getsentry/sentry-dotnet/actions/runs/31783003688/job/94712772156
- 8.54.0, .NET (win-x64) 2026-08-28 — https://github.com/getsentry/sentry-dotnet/actions/runs/33145322315/job/98765034116

```
error MSB3923: Failed to download file
"https://repo1.maven.org/maven2/io/sentry/sentry-android-ndk/8.54.0/sentry-android-ndk-8.54.0.pom".
Response status code does not indicate success: 403 (Forbidden).
```

The download already sets `Retries="3"`:

https://github.com/getsentry/sentry-dotnet/blob/main/src/Sentry.Bindings.Android/Sentry.Bindings.Android.csproj#L139-L145

but that is a no-op here, because MSBuild's `DownloadFile` treats only **500** and **408**
(plus mid-transfer IO errors) as retriable — a 403 is never retried. See
`IsRetriable` in https://github.com/dotnet/msbuild/blob/main/src/Tasks/DownloadFile.cs.

The logs confirm it empirically: on the 8.54.0 failure, cache extraction at `05:42:35`,
download failure at `05:42:41`, job dead at `05:42:43` — a single attempt, no backoff.

This request is also completely unprotected: `SentryAndroidSdkDirectory` is under
`$(BaseIntermediateOutputPath)` and nothing in `build.yml` caches it (the `actions/cache`
steps are all for sentry-native), so **every** Android-targeting CI build makes exactly one
un-retried request to `repo1.maven.org`.

**NuGet.org `503`:**
- MSBuild 2026-08-21 — https://github.com/getsentry/sentry-dotnet/actions/runs/32462737597/job/96712991512

```
NuGet.targets(198,5): error : Response status code does not indicate success: 503 (Service Unavailable).
```

Note `DownloadFile` does not retry **503** either, so the same latent hole covers both
status codes actually seen in the wild.

**Proposed fix** for the Maven case — retry across attempts regardless of status code, with
a mirror fallback (both mirrors verified reachable):

```xml

<_NdkPom>$(SentryAndroidSdkDirectory)sentry-android-ndk-$(SentryAndroidSdkVersion).pom
<_NdkPomPath>io/sentry/sentry-android-ndk/$(SentryAndroidSdkVersion)/sentry-android-ndk-$(SentryAndroidSdkVersion).pom

```

`ContinueOnError="WarnAndContinue"` is safe here: `Directory.Build.props` sets
`TreatWarningsAsErrors` (the *compiler* property), not `MSBuildTreatWarningsAsErrors`, and CI
does not pass `-warnaserror`.

For the NuGet 503, it is worth checking whether `NUGET_ENABLE_ENHANCED_HTTP_RETRY` is in
effect for these jobs — that has not been verified.

An optional second layer for both: cache the Android SDK's `MavenCacheDirectory` (it is
redirectable via the `MavenCacheDirectory` property), which would take Maven Central off the
steady-state build path entirely. Note it would still miss on version-bump PRs, which is
where the risk is highest.

---

### 3. Integration test race (2 hits)

```
Expected string '"message":"Hello from MSBuild app"' to match any element in collection @()
```

The mock Sentry server received **nothing** — the collection is empty, so the assertion runs
before the envelope arrives (or the app never flushed). Needs a poll/wait rather than an
immediate assert.

- MSBuild 2026-08-24 — https://github.com/getsentry/sentry-dotnet/actions/runs/32707533251/job/97371911312
- .NET (win-x64) 2026-08-14 — https://github.com/getsentry/sentry-dotnet/actions/runs/31780243320/job/94704372605

---

### 4. Unclassified (2 hits)

- .NET (linux-arm64) 2026-08-18, `Test` step — the log ends with `Terminate orphan process: pid (22864) (dotnet)` x3 plus a `VBCSCompiler`, which suggests a hung or crashed test host, but there is no explicit failure message to go on.
- .NET (linux-musl-x64) 2026-08-13 — log has aged out of retention.

## Suggested order of work

1. `-m:1` on the pack step — 4 hits, one line, fixes all platforms.
2. Maven retry + mirror fallback — 3 hits, self-contained csproj change.
3. Integration test wait loop — 2 hits, needs a real look at the assertion.

Deliberately **not** proposing a job-level retry action (e.g. `nick-fields/retry`) as the
primary fix: it re-runs a ~5 minute build to work around a 4 KB download, and it would mask
genuine failures. Recent real (non-flaky) failures in the same window included a `CS0426`
Android binding breakage, a Verify snapshot mismatch and a device-test failure — all of which
should stay red.

Start with .github/workflows/build.yml and reproduce or inspect the Create NuGet Packages step, then read the download logic in src/Sentry.Bindings.Android/Sentry.Bindings.Android.csproj. Review the integration test assertion involved in the empty collection failure. Done means the identified CI races are addressed, transient dependency failures are handled, and genuine build or test failures still remain visible.

Written by the indexing model from the issue text.