dotnet / dotnet/macios

[CI] Expired Apple development certificates break signing-dependent PR tests

Open
#26,382 18 comments 0 reactions 0 assignees View on GitHub
ci-failure ci-postmortem tests
Dominant language
C#
Stars
2.9k
Forks
576
Avg merge
2d 13h
Merged PRs (30d)
96

Description

## Summary

The Apple development certificates installed in the PR-test signing keychains expired on July 31, 2026. Signing-dependent tests now fail with:

`No valid iOS/tvOS/macOS code signing keys found in keychain.`

- Stable signature: `expired-apple-development-certificates-no-valid-code-signing-keys`
- Classification: CI infrastructure failure
- Distinct monitored PRs: 4
- Confirmed job occurrences: 17
- First confirmed by the monitor: 2026-07-31T14:42:07Z
- Latest confirmed occurrence: 2026-07-31T16:30:04Z

## Affected PRs

- #26297, head `7e56322bf7c853b1c4ef12626f02cccf36d26fe5`
- [Build 14833205](https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=14833205), attempt 1
- Failed jobs: `dotnettests_ios`, `dotnettests_tvos`, `dotnettests_macos`, `monotouch_ios`, `monotouch_tvos`, `msbuild`, and Windows `Dotnet tests`
- #26281, head `2df8ff4002b2b5a6d461eeb47a45b9bf576130df`
- [Build 14834283](https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=14834283), attempt 1
- Failed job: [`msbuild`](https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=14834283&view=logs&jobId=43070558-71ad-5208-aa3c-4b38cce0e88e)
- #26380, head `85c0366c6c1b1233983943d0d59f08633b291c3c`
- [Build 14834129](https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=14834129), attempt 1
- Failed jobs: [`dotnettests_tvos`](https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=14834129&view=logs&jobId=09d26b41-67aa-54a4-f0c7-0e78dc94babc), [`dotnettests_macos`](https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=14834129&view=logs&jobId=306fc6d1-8bac-5ab5-4f68-7aa08eedb9c6), [`monotouch_ios`](https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=14834129&view=logs&jobId=a70ed34c-97c9-5ebb-c5be-8e2604f5ffcc), [`monotouch_tvos`](https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=14834129&view=logs&jobId=1acdac53-199e-5330-5c52-bf5d9d7be24b), and [`msbuild`](https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=14834129&view=logs&jobId=43070558-71ad-5208-aa3c-4b38cce0e88e)
- #26381, head `bf31c1874c3da70065394e2cf9a8feadbf722c34`
- [Build 14834130](https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=14834130), attempt 1
- Failed jobs: [`dotnettests_tvos`](https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=14834130&view=logs&jobId=09d26b41-67aa-54a4-f0c7-0e78dc94babc), [`monotouch_ios`](https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=14834130&view=logs&jobId=a70ed34c-97c9-5ebb-c5be-8e2604f5ffcc), [`monotouch_tvos`](https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=14834130&view=logs&jobId=1acdac53-199e-5330-5c52-bf5d9d7be24b), and [`msbuild`](https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=14834130&view=logs&jobId=43070558-71ad-5208-aa3c-4b38cce0e88e)

## Diagnostic evidence

- `DetectSigningIdentityTaskTests` fails because no valid iOS signing key remains.
- Detailed MSBuild test reports confirm that the installed Apple development certificates are expired.
- iOS, tvOS, and macOS app-build tests then fail in signing detection; the later test-suite summaries are consequences of that same root cause.
- The first failed task in each listed job is `Run tests`. No publication or missing-artifact failure is classified separately.

## Reproduction pattern and impact

Any PR test job that builds a signing-dependent device or extension configuration after the certificate expiry can fail. The issue currently blocks four monitored PRs and affects simulator, Windows-remote, MSBuild, and app-build test jobs across iOS, tvOS, and macOS.

## Investigation performed

The Azure timelines, TestSummary reports, detailed NUnit reports, and representative binlogs were inspected. Searches across open and closed `dotnet/macios` issues used the exact error, `DetectSigningIdentityTaskTests`, keychain-signing terms, and expired-certificate terms. Issues #7171 and #20671 contain the same user-facing error but describe unrelated project/configuration problems, not expired CI credentials.

## Retry status

- #26297: no retry submitted. The durable budget is 3 lifetime retries and 0 for the current head.
- #26281, #26380, and #26381: no retry submitted; their initial builds are still active.
- Retrying before replacement credentials are validated would deterministically repeat the failure.

## Resolution / workaround

Renew and replace the Apple development certificates and associated provisioning profiles in the secure CI signing bundle/keychains. Validate the replacement with a full signing-dependent simulator run. After validation, rerun failed jobs; if Windows Integration is affected, rerun its entire stage.

Azure artifacts and detailed reports are retention-limited.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the DetectSigningIdentityTaskTests failures and the Azure timelines, TestSummary reports, detailed NUnit reports, and representative binlogs named in the issue. Validate replacement signing credentials with a full signing-dependent simulator run, then rerun the affected jobs and confirm the iOS, tvOS, macOS, MSBuild, and Windows test failures are resolved.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
ci-cd, devops, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.