dotnet / dotnet/arcade-services
Categorize timeout failures for prioritized investigation
- Dominant language
- C#
- Stars
- 86
- Forks
- 86
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 35
Description
## Business value
Help .NET engineers prioritize recurring CI timeout investigations instead of repeatedly triaging isolated failures.
## Expected outcome
Understandable timeout categories with trustworthy impact statistics and inspectable evidence, available in the triage workflow.
## Acceptance criteria
- [ ] Reviewed examples demonstrate useful grouping of supported timeout failures. Categories describe observed patterns, not unproven root causes.
- [ ] Categories link representative failures. Uncertain and uncategorized timeouts remain counted and reviewable. Existing visibility rules apply to summaries and evidence.
- [ ] Report occurrences and distinct affected builds separately: three timeouts in one build mean three occurrences and one affected build. Reprocessing does not inflate counts; overall build totals deduplicate across categories.
- [ ] Show frequency, trends and affected repositories/platforms for a stated period and population, with data freshness. Flag missing data or coverage/category changes that invalidate comparisons.
- [ ] From the agreed triage entry point, reviewers can compare category frequency, build impact and trends, then inspect examples.
- [ ] Document which summaries and evidence non-UI consumers need, and whether access is delivered here or through related data-access work.
## Scope
**In:** Categorization, impact statistics and evidence-backed triage as specified above.
**Out:** Timeout fixes, known-issue automation, retry/merge-policy changes, the complete Data API and unbounded historical backfill. No particular AI technique or architecture is required. Keep specifications inline; consider decomposition only after prioritization.
## Dependencies and prerequisites
Confirm evidence, identifiers, metadata and access in the [current implementation](https://dnceng.visualstudio.com/internal/_git/dotnet-build-insights). Agree initial coverage, counting rules and representative validation examples.
## Risks and mitigations
Misleading grouping, duplicate counts and incomplete coverage can distort priorities; review representative edge cases and disclose limitations. Restrict scope if usable evidence or permissions are insufficient.
## Delivery assessment
> Delivery assessment is **experimental**; estimates and AI-suitability ratings are provisional.
- **Size:** `XL` (5+ engineering days); refine and consider later decomposition after prioritization.
- **Complexity drivers:** Category quality, aggregation, evidence access and triage integration.
- **AI implementation suitability:** `Medium` - coding assistance is useful, but category judgment and data/access decisions need people.
- **Human checkpoints:** Approve coverage, counting semantics, category examples, design and visibility.
- **Assessment assumptions:** Current implementation coverage and telemetry suitability are unverified; no historical-effort comparison. Re-estimate after refinement.
## Related
- [dotnet/arcade-services#6584](https://github.com/dotnet/arcade-services/issues/6584): known-issue automation, a separate follow-on.
- [dotnet/arcade-services#6585](https://github.com/dotnet/arcade-services/issues/6585): public/outerloop definitions and independent statistics.
- [dotnet/arcade-services#6140](https://github.com/dotnet/arcade-services/issues/6140): overlapping original proposal; design ideas are not requirements.
- [dotnet/build-insights#61](https://github.com/dotnet/build-insights/issues/61): known-issues list with occurrence/build statistics.
- [dotnet/build-insights#77](https://github.com/dotnet/build-insights/issues/77): Data API for UI, individual and AI-tool consumers.
- [dotnet/build-insights#73](https://github.com/dotnet/build-insights/issues/73): documented, versioned programmatic BI-data access.
The three Build Insights links are historical; migrated counterparts remain unidentified. Legacy closures reflect cleanup, not delivery. Related platform features are not automatically blockers.
## Open decisions
- Initial timeout/build populations, reporting windows, retry/rerun counting and bounded backfill.
- Category quality thresholds and lifecycle, including merge/split effects on trends.
- Triage entry point, freshness and non-UI delivery scope.
---
🤖 _Drafted by an AI agent on behalf of @michalpavelka._
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the linked current implementation in dotnet-build-insights and confirm the available timeout evidence, identifiers, metadata, and access. Resolve the initial populations, counting rules, category validation, reporting window, and triage entry point before implementation. Done means reviewers can compare categorized frequency, affected builds, trends, and representative evidence without duplicate counts or hidden coverage limits.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- devops, observability-sre
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100