dotnet / dotnet/macios

[CI] xcodebuild -downloadPlatform tvOS hangs until test-job timeout

Open
#26,533 2 comments 0 reactions 0 assignees View on GitHub
ci-failure ci-postmortem
Dominant language
C#
Stars
2.9k
Forks
576
Avg merge
2d 13h
Merged PRs (30d)
96

Description

## Failure signature

`Provision Xcode and dependencies` invokes `xcodebuild -downloadPlatform tvOS`, which then produces no output for more than 2.5 hours and consumes the test job's 180-minute budget.

Stable signature:

```text
Provision Xcode xcodebuild -downloadPlatform tvOS hangs after invocation for over 2.5 hours and exhausts the 180-minute test-job budget
```

## Confirmed occurrences

### PR #26527

- Head: `73d2ac20d31eecde56341c51d7d83b2da3c3ab20`
- [Build 15181280, `T: introspection` / Provision Xcode and dependencies](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15181280&view=logs&j=6e00b7cd-b8a5-532f-afa5-bfd80bdfa50e&t=b0160d47-4cd4-5425-1852-3d44b890a345)
- Worker: `AcesShared 182`
- Xcode 27 beta 6 command: `xcodebuild -downloadPlatform tvOS -architectureVariant arm64`
- The command produced no output for **9,376 seconds**. Provisioning eventually completed after 9,950 seconds, but only eight minutes remained before Azure canceled the job at 180 minutes; introspection did not complete.

### PR #26240

- Head: `69ff5adf1625c166175164e1c040e45d2defca35`
- [Build 15181530, `T: framework` / Provision Xcode and dependencies](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15181530&view=logs&j=456345f1-e486-5e62-b2a7-0064843ff30e&t=13736b78-2418-5eb2-9b4d-44ff165b83ba)
- Worker: `AcesShared 90`
- Xcode 26.6 command: `xcodebuild -downloadPlatform tvOS -architectureVariant universal`
- The command produced no output for **10,442 seconds** until Azure canceled the job. Tests never started.

First observed: `2026-09-02T07:40:37Z`.
Latest confirmed occurrence: `2026-09-02T11:05:24Z`.
Recurrence: **2 jobs across 2 distinct monitored PRs**.

## Classification and impact

This is a CI infrastructure failure in Xcode platform provisioning, not a product/test assertion or API-diff failure. It reproduces with different Xcode versions, architecture variants, workers, PRs, and test jobs. The silent platform-download process consumes the entire job budget and prevents tests from running or completing.

## Investigation performed

- Inspected both Azure timelines, task logs, worker assignments, and exact task intervals.
- Calculated the longest timestamp gaps directly from each retained provisioning log.
- Searched all open and closed `dotnet/macios` issues separately using the exact command, task name, architecture variants, `Finding content`, and job-timeout symptoms.
- Rejected #26221 as unrelated: it tracks an iOS runtime test/NSUrlSession stall after tests launch, while these failures occur during Xcode provisioning.

## Workaround

Rerun the failed simulator jobs after the run is terminal and retry/merge gates permit it. The durable fix should bound or cancel the `xcodebuild -downloadPlatform` subprocess independently so one platform discovery cannot consume the full test-job budget.

Azure task logs are retention-limited.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the CI task named “Provision Xcode and dependencies” and trace the xcodebuild -downloadPlatform tvOS invocation in the affected simulator jobs. Reproduce or inspect the retained provisioning logs, then verify that a stalled platform download is bounded or canceled independently and that the test job can proceed before its 180-minute timeout.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, macos
Domain
ci-cd, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.