[CI] macOS checkout fetch ends with early EOF and loses credentials on retry
- Dominant language
- C#
- Stars
- 2.9k
- Forks
- 576
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 96
Description
## Summary
Two macOS Sonoma checkout jobs failed before repository build or test tasks with the same Git transfer and retry-authentication sequence.
- Stable signature: `git-fetch-early-eof-credentials-lost`
- Classification: CI infrastructure failure
- Distinct monitored PRs: 2
- Confirmed job occurrences: 2
- First confirmed by the monitor: 2026-07-22T15:59:55Z
- Latest confirmed occurrence: 2026-08-01T03:43:24Z
## Occurrences
- #26235, head `2d6de8eaf12aff5c2f4bff984ce5085b1fece6f7`
- [Build 14731906, attempt 1, X64 Mac Sonoma job](https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=14731906&view=logs&jobId=623bc498-120b-5ff2-984a-5b073efffe1b)
- Checkout failed with an unexpected sideband disconnect, `fatal: early EOF`, and `invalid index-pack output`; task retries then could not read a GitHub username because terminal prompts were disabled.
- Retry outcome: Sonoma attempt 2 succeeded at 2026-07-22T16:29:53Z. The retained timeline also shows a later Sonoma attempt 3 succeeding.
- #26384, head `d45f335fdc821987a0d0086ebd976c6efb545806`
- [Build 14840799, attempt 1, X64 Mac Sonoma job](https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=14840799&view=logs&jobId=623bc498-120b-5ff2-984a-5b073efffe1b)
- Checkout fetched for about 62 minutes, then logged `unexpected disconnect while reading sideband packet`, `fatal: early EOF`, and `fatal: fetch-pack: invalid index-pack output` in task log 2883.
- All task-level retries then failed with `could not read Username for https://github.com: terminal prompts disabled`.
- Build 14840799 completed failed at 2026-08-01T03:47:30Z.
## Reproduction pattern and impact
The macOS checkout performs a large full-ref fetch. When that transfer disconnects after a long download, the retry path reinitializes the repository but no longer has usable GitHub credentials. The job never reaches repository build or test tasks.
## Investigation performed
The Azure build metadata, timelines, checkout task logs, and retry outcomes were inspected. Separate all-state issue searches used the exact sideband-disconnect, early-EOF, invalid-index-pack, credential, and checkout component strings. The only candidate, #10251, concerns product build performance and is unrelated.
No TestSummary or HtmlReport diagnostic is applicable because checkout failed before tests started.
## Retry status
- #26235 recovered on a later Sonoma retry.
- #26384 was not retried. Its completed run also contains deterministic expired-certificate failures, including Windows Integration, and replacement signing credentials are not validated. Retrying the failed set would repeat those failures, and Windows Integration requires an entire-stage retry.
## Current diagnosis and workaround
This is a CI checkout/network failure followed by a broken retry-authentication path. Investigate Git transfer reliability for the macOS agent and ensure checkout retries preserve or reacquire the GitHub authentication header. A fresh checkout attempt can recover, but #26384 must not be retried until the separate signing credentials are validated.
Azure logs and artifacts are retention-limited.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the Azure build metadata and checkout task logs for builds 14731906 and 14840799, focusing on the macOS full-ref fetch and its retry path. Trace how the retry handles the GitHub authentication header after early EOF and invalid index-pack errors. Done means the checkout retry preserves or reacquires authentication and the failure pattern is addressed without retrying the unrelated signing-credential failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, git, github, macos
- Domain
- ci-cd, devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100