dotnet / dotnet/sdk

[AI discovered CI] dotnet-new integration work item repeatedly times out during template commands

Open
#56,281 1 comment 1 reaction 2 assignees Claimed by @nagilson View on GitHub
agentic-workflows cookie live-build-incident Test Debt untriaged
Dominant language
C#
Stars
3.2k
Forks
1.3k
PR merge metrics
PR metrics pending

Description

## Build Information
- Current build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=1594166
- Branch: `refs/heads/main`
- Failing work item: `dotnet-new.IntegrationTests.dll.1` in the Linux x64 - ubuntu.2204.amd64.open queue
- Phase: `test-execution`
- Failure type: `timeout`
- Evidence sources: `helix-console`, `process-exit-code`
- **Failure fingerprint:** `test-execution|timeout|dotnet-new.integrationtests.dll.|command-dotnet-build-exited-with-exit-code-.-command-dotnet-new-classlib--o----name-classlib---langversion-.---framework-netstandard2.---debug-custom-hi`
- Matching prior builds: [20260911.32 / build 1592831](https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=1592831), [20260911.2 / build 1592304](https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=1592304)

## Failure History
The supplied evidence contains three matching `dotnet-new.IntegrationTests.dll.1` timeout occurrences: builds 1594166, 1592831, and 1592304, from independent commits. The surrounding sequence includes successful builds 1592979 and 1592713 between matching failures. The related failures used different template command variants, but the stable observed mechanism is the same Linux `dotnet-new` integration work item exceeding its execution limit. This establishes recurrence of the timeout pattern, not the underlying cause.

## Error Details
Exact current observation excerpt:

> Command `dotnet build` exited with exit code 0.
> Command `dotnet new classlib -o --name ClassLib --langVersion 9.0 --framework netstandard2.0 --debug:custom-hive ...` exited with exit code 0.
> `dotnet-new.IntegrationTests.dll.1` END OF WORK ITEM LOG: Command timed out, and was killed

The current work item exit code was `-3` and its duration was `01:00:37.3850000`. Console log: https://helix.dot.net/api/jobs/8224ee87-9326-4efa-801c-a14bd44cae38/workitems/dotnet-new.IntegrationTests.dll.1/files/console.885d1b84.log?api-version=2019-06-17. No dump or additional result artifact was available in the supplied evidence.

## Root Cause Analysis
**Observed** The Linux `dotnet-new` integration work item completes several `dotnet build` and `dotnet new classlib` commands successfully, then is killed after the work-item timeout.

**Assessment** The immediate cause of the red pipeline is that this SDK-owned integration-test work item does not finish within the Helix execution limit. Recurrence across independent builds makes this a sustained CI quality problem requiring investigation. The evidence does not establish whether the delay is a test-code hang, an SDK/template regression, or environment-dependent resource contention. The generic `Monitor Helix Jobs` failure is a downstream report, not the cause.

**Confidence** Medium. The timeout and affected work item are directly established and recurring, but no dump or timing breakdown identifies the underlying stall.

**Alternatives / Unknowns** The failing command is only the last command visible in the excerpt; the actual blocked operation may be later template coverage. A Linux queue/resource issue, a change in template test setup, or a gradual performance regression remain possible. The dossier contains no target-branch comparison, changed-file correlation, binlog, or dump analysis.

## Suggested Investigation
1. Capture a dump or detailed per-test/per-command timing from a reproduction of `dotnet-new.IntegrationTests.dll.1` on the Linux queue, identifying the operation still active at timeout.
2. Compare the test and SDK/template changes between the last passing build and build 1592831/1592304, then inspect the affected integration-test loop and custom-hive/process cleanup.
3. Run the focused integration test with diagnostic logging and compare resource usage and duration against a passing build; determine whether extending or partitioning the work item would mask a test hang or address a genuine performance regression.

> Generated by [🕵️ CI Quality Investigator](https://github.com/dotnet/sdk/actions/runs/34738108507) · gpt56 · 1.53 AIC · ⌖ 0.801 AIC · ⊞ 31.8K · [◷](https://github.com/search?q=repo%3Adotnet%2Fsdk+is%3Aissue+%22gh-aw-workflow-call-id%3A+dotnet%2Fsdk%2Fci-quality-monitor%22&type=issues)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.