microsoft / microsoft/aspire

Flaky test: VerifyYarpResourceExtensionsConfig - backend resource fails to become healthy on ubuntu-latest (DCP managedFields error)

Open
#17,020 2 comments 0 reactions 1 assignee Claimed by @danegsta View on GitHub
area-orchestrator flaky-test quarantined-test triage:bot-seen yarp
Dominant language
C#
Stars
6.3k
Forks
991
Avg merge
2d 15h
Merged PRs (30d)
196

Description

## Summary

`Aspire.Hosting.Yarp.Tests.YarpFunctionalTests.VerifyYarpResourceExtensionsConfig` has been failing continuously on the `Tests / Hosting.Yarp / Hosting.Yarp (ubuntu-latest)` CI job. The same failure reproduced **6 times across all 6 attempts** of CI run [`25774478514`](https://github.com/microsoft/aspire/actions/runs/25774478514) on PR #16983, with an identical root cause every time.

This test has been re-quarantined while we investigate. See the prior flake history in #9344 — the test was unquarantined in March 2026 and has now started failing again.

Per Karol, we should consider pulling in the latest DCP dependencies. The DCP version currently pinned in [`eng/Versions.props`](https://github.com/microsoft/aspire/blob/main/eng/Versions.props) is `0.23.5`, and the failure signature points squarely at DCP internals.

## Reproducer

- Failing job (run 25774478514, attempt 6, ubuntu-latest): https://github.com/microsoft/aspire/actions/runs/25774478514/job/75823111198?pr=16983
- Failing test step / log line: https://github.com/microsoft/aspire/actions/runs/25774478514/job/75823111198?pr=16983#step:31:51
- **Attempts: 6 of 6 failed for the same reason.** The corresponding `Hosting.Yarp (windows-latest)` job passed on every attempt — failure is ubuntu-latest only.
- Aspire build under test: `13.4.0-pr.16983.g71822d92`

## Failure signature

```
failed Aspire.Hosting.Yarp.Tests.YarpFunctionalTests.VerifyYarpResourceExtensionsConfig (19s 560ms)
Xunit.MicrosoftTestingPlatform.XunitException:
Aspire.Hosting.DistributedApplicationException : Stopped waiting for resource 'backend'
to become healthy because it failed to start.
at Aspire.Hosting.ApplicationModel.ResourceNotificationService.WaitForResourceHealthyAsync(...)
in /_/src/Aspire.Hosting/ApplicationModel/ResourceNotificationService.cs:250
at Aspire.Hosting.Yarp.Tests.YarpFunctionalTests.VerifyYarpResource(...)
in tests/Aspire.Hosting.Yarp.Tests/YarpFunctionalTests.cs:48
at Aspire.Hosting.Yarp.Tests.YarpFunctionalTests.VerifyYarpResourceExtensionsConfig()
in tests/Aspire.Hosting.Yarp.Tests/YarpFunctionalTests.cs:17
Test run summary: Failed!
```

DCP debug log from the failing run shows an internal `[SHOULD NOT HAPPEN]` error reconciling the `backend` service:

```
Aspire.Hosting.Dcp.dcp Debug: [SHOULD NOT HAPPEN] failed to update managedFields
{"versionKind": "usvc-dev.developer.microsoft.com/v1, Kind=Service",
"namespace": "", "name": "backend",
"error": "failed to convert new object (/backend; usvc-dev.developer.microsoft.com/v1, Kind=Service)
to smd typed: no corresponding type for usvc-dev.developer.microsoft.com/v1, Kind=Service"}
```

The `backend` container never reaches a `Healthy` state, so the test times out waiting on it; the YARP route subsequently can't proxy. This explains the older `BadGateway` symptom from #9344 — the upstream is just never reachable.

## Suggested next step

1. Update the DCP versions in `eng/Versions.props` (`MicrosoftDeveloperControlPlane*Version`, currently `0.23.5`) to the latest available release.
2. Re-run the Yarp tests against the new DCP and verify `VerifyYarpResourceExtensionsConfig` is stable.
3. Once stable, unquarantine the test.

## Related

- Prior flake issue: #9344
- PR where this was observed: #16983

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.