dotnet / dotnet/runtime

Test failure: PingTest.SendPingWithLowTtl_RoundtripTimeIsNonZero flaky on macOS 26 ARM64 (sub-ms RTT truncated to 0)

Open Beginner friendly
#132,123 1 comment 0 reactions 0 assignees View on GitHub
area-System.Net Known Build Error os-mac-os-x
Dominant language
C#
Stars
18.3k
Forks
5.6k
PR merge metrics
PR metrics pending

Description

## Test failure

`System.Net.NetworkInformation.Tests.PingTest.SendPingWithLowTtl_RoundtripTimeIsNonZero` is failing in outerloop on **macOS 26 ARM64** (`OSX.26.Arm64.Open`), starting **2026-08-09**.

### Error message

```
Expected at least one TtlExpired reply with non-zero RoundtripTime across 3 attempts
```

### Frequency

2 builds on **main** over the past 7 days, exclusively on the `osx.26.arm64.open` queue (`runtime-libraries-coreclr outerloop-osx`). Not seen on any other platform.

## Root cause

Environment/platform-specific test fragility, not a product bug.

The test (`src/libraries/System.Net.Ping/tests/FunctionalTests/PingTest.cs`) pings with `Ttl = 5`, expects a `TtlExpired`/`TimeExceeded` reply, and asserts at least one of 3 attempts reports a non-zero `RoundtripTime`:

```csharp
// On macOS, the raw socket implementation truncates sub-millisecond RTT to 0,
// and the first-hop router often responds in <1ms.
options.Ttl = 5;
bool gotNonZeroRtt = false;
for (int attempt = 0; attempt < 3; attempt++)
{
PingReply pingReply = await ping.SendPingAsync(host, TestSettings.PingTimeout, payload, options);
Assert.True(pingReply.Status == IPStatus.TimeExceeded || pingReply.Status == IPStatus.TtlExpired, ...);
if (pingReply.RoundtripTime > 0) { gotNonZeroRtt = true; break; }
}
Assert.True(gotNonZeroRtt, "Expected at least one TtlExpired reply with non-zero RoundtripTime across 3 attempts");
```

The test comment already acknowledges the failure mode: macOS raw sockets truncate sub-millisecond RTT to 0. On the fast Azure `OSX.26.Arm64` CI runners the first-hop router consistently replies in under 1ms, so all 3 attempts measure `RoundtripTime == 0` and the assertion fails. This is inherently timing/environment dependent and will flake on fast, low-latency network paths.

## Suggested fix

Make the test robust to sub-millisecond RTT on macOS, e.g.:
- Skip the non-zero-RTT assertion on macOS (the RTT-truncation platform), or
- Increase the number of attempts and/or accept `RoundtripTime >= 0` on platforms known to truncate sub-ms RTT, keeping the strict `> 0` assertion only where the raw-socket path preserves sub-ms timing.

## Sample console log

https://helixr1107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-c2d1c1347a4940c8a6/System.Net.Ping.Functional.Tests/3/console.ff3a2daa.log?helixlogtype=result

## Known issue error message

Fill the error message using [known issues](https://github.com/dotnet/arcade/blob/main/Documentation/Projects/Build%20Analysis/KnownIssues.md).

```json
{
"ErrorMessage": "Expected at least one TtlExpired reply with non-zero RoundtripTime across 3 attempts",
"BuildRetry": false,
"ExcludeConsoleLog": false
}
```

### Known issue validation
**Build: :mag_right:** https://dev.azure.com/dnceng-public/public/_build/results?buildId=1546596
**Error message validated:** `[Expected at least one TtlExpired reply with non-zero RoundtripTime across 3 attempts`]
**Result validation:** :white_check_mark: Known issue matched with the provided build.
**Validation performed at:** 8/11/2026 9:19:16 AM UTC

### Report
#### Summary
|24-Hour Hit Count|7-Day Hit Count|1-Month Count|
|---|---|---|
|0|0|0|

Contributor guide

Open the contributing guide

Research direction

Start in src/libraries/System.Net.Ping/tests/FunctionalTests/PingTest.cs at PingTest.SendPingWithLowTtl_RoundtripTimeIsNonZero and read the existing macOS RTT comment. Run the test on the OSX.26.Arm64.Open queue and verify the change handles sub-millisecond RoundtripTime values without weakening the expected TTL-expired response checks or allowing the outerloop failure to recur.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, macos
Domain
ci-cd, networking, testing-qa
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.