dotnet / dotnet/macios

[CI] Ventura Show Bot Info stalls until the 120-minute job timeout

Open
#26,457 32 comments 0 reactions 0 assignees View on GitHub
ci-postmortem copilot
Dominant language
C#
Stars
2.9k
Forks
576
Avg merge
2d 12h
Merged PRs (30d)
123

Description

## Failure signature

The **M1 - Mac Ventura (13) macOS tests** job stalls in the pre-test `Show Bot Info` task on `VSM-XAM-70.Ventura.arm64` until Azure DevOps cancels the job:

```text
The job running on agent VSM-XAM-70.Ventura.arm64 ran longer than the maximum time of 120 minutes.
The Operation will be canceled.
```

All five occurrences use job `e59aa738-7d46-5a5e-8f43-c6b31f5d2976` and task `fdc1737c-f64b-5ec2-e381-d3e861748b97`, both at attempt 1. No tests start.

## Confirmed occurrences

| PR / head | Azure run | Terminal time (UTC) | Task |
|---|---|---|---|
| #26437 / `f0243b8d39cefffaf2c857b7bf0713331c111b00` | [15030991 (`20260820.1`)](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15030991) | 2026-08-20 04:47:56 | [Show Bot Info](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15030991&view=logs&j=e59aa738-7d46-5a5e-8f43-c6b31f5d2976&t=fdc1737c-f64b-5ec2-e381-d3e861748b97) |
| #26437 / `3fb3386245d841672f793bead4bf8c56f3d70149` | [15016000 (`20260819.2`)](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15016000) | 2026-08-19 04:44:42 | [Show Bot Info](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15016000&view=logs&j=e59aa738-7d46-5a5e-8f43-c6b31f5d2976&t=fdc1737c-f64b-5ec2-e381-d3e861748b97) |
| #26444 / `67e16a518ef18a296c6c9040021675b35ead479e` | [15063623 (`20260823.3`)](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15063623) | 2026-08-23 04:41:17 | [Show Bot Info](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15063623&view=logs&j=e59aa738-7d46-5a5e-8f43-c6b31f5d2976&t=fdc1737c-f64b-5ec2-e381-d3e861748b97) |
| #26444 / `7e32119e9ac03e47fbdb101036d93d683ff67a57` | [15045113 (`20260821.6`)](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15045113) | 2026-08-21 04:47:37 | [Show Bot Info](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15045113&view=logs&j=e59aa738-7d46-5a5e-8f43-c6b31f5d2976&t=fdc1737c-f64b-5ec2-e381-d3e861748b97) |
| #26444 / `e28113097531b53792a97564a04a949b74658aa6` | [15000705 (`20260818.3`)](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15000705) | 2026-08-18 04:42:57 | [Show Bot Info](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15000705&view=logs&j=e59aa738-7d46-5a5e-8f43-c6b31f5d2976&t=fdc1737c-f64b-5ec2-e381-d3e861748b97) |

**Total:** 5 occurrences across 2 distinct monitored PRs and 5 distinct heads. First observed at `2026-08-18T04:42:57Z`; latest observed at `2026-08-23T04:41:17Z`.

## Classification and impact

**CI infrastructure failure.** The same pre-test task, worker, record IDs, and hard timeout recur on unrelated dependency-update PRs. The job never reaches build or test execution, so this is not evidence of a source or deterministic test regression. It blocks the complete Ventura stage and the overall simulator check.

## Investigation performed

- Inspected authenticated Azure DevOps build metadata, trigger fields, timelines, task logs, attempts, and retained run history.
- Verified each occurrence was an automatic initial PR run on a distinct head, with attempt 1 and no previous attempts.
- Correlated 5 exact-signature occurrences among 15 retained simulator runs for these PRs.
- Searched open and closed `dotnet/macios` issues separately by exact timeout text, task name, worker, and Ventura component terms. Existing timeout issues concern test execution or provisioning and do not match this pre-test task stall.

The lower-level reason `Show Bot Info` stops making progress remains unresolved. Azure task logs are retention-limited.

## Workaround

Rerun failed jobs after refreshing the PR head, merge-conflict state, retry budgets, and equivalent active attempts. This may unblock the PR but does not fix the affected worker/task.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the authenticated Azure DevOps build metadata, timelines, and retained logs for the Ventura job's pre-test Show Bot Info task, comparing the five listed runs and worker VSM-XAM-70.Ventura.arm64. Determine why that task stops progressing and verify the affected Ventura stage completes without the 120-minute timeout; the issue notes that task logs are retention-limited.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
ci-cd, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.