Test failure: PingTest.SendPingWithLowTtl_RoundtripTimeIsNonZero flaky on macOS 26 ARM64 (sub-ms RTT truncated to 0)
- Dominant language
- C#
- Stars
- 18.3k
- Forks
- 5.6k
- PR merge metrics
- PR metrics pending
Description
## Test failure
`System.Net.NetworkInformation.Tests.PingTest.SendPingWithLowTtl_RoundtripTimeIsNonZero` is failing in outerloop on **macOS 26 ARM64** (`OSX.26.Arm64.Open`), starting **2026-08-09**.
### Error message
```
Expected at least one TtlExpired reply with non-zero RoundtripTime across 3 attempts
```
### Frequency
2 builds on **main** over the past 7 days, exclusively on the `osx.26.arm64.open` queue (`runtime-libraries-coreclr outerloop-osx`). Not seen on any other platform.
## Root cause
Environment/platform-specific test fragility, not a product bug.
The test (`src/libraries/System.Net.Ping/tests/FunctionalTests/PingTest.cs`) pings with `Ttl = 5`, expects a `TtlExpired`/`TimeExceeded` reply, and asserts at least one of 3 attempts reports a non-zero `RoundtripTime`:
```csharp
// On macOS, the raw socket implementation truncates sub-millisecond RTT to 0,
// and the first-hop router often responds in <1ms.
options.Ttl = 5;
bool gotNonZeroRtt = false;
for (int attempt = 0; attempt < 3; attempt++)
{
PingReply pingReply = await ping.SendPingAsync(host, TestSettings.PingTimeout, payload, options);
Assert.True(pingReply.Status == IPStatus.TimeExceeded || pingReply.Status == IPStatus.TtlExpired, ...);
if (pingReply.RoundtripTime > 0) { gotNonZeroRtt = true; break; }
}
Assert.True(gotNonZeroRtt, "Expected at least one TtlExpired reply with non-zero RoundtripTime across 3 attempts");
```
The test comment already acknowledges the failure mode: macOS raw sockets truncate sub-millisecond RTT to 0. On the fast Azure `OSX.26.Arm64` CI runners the first-hop router consistently replies in under 1ms, so all 3 attempts measure `RoundtripTime == 0` and the assertion fails. This is inherently timing/environment dependent and will flake on fast, low-latency network paths.
## Suggested fix
Make the test robust to sub-millisecond RTT on macOS, e.g.:
- Skip the non-zero-RTT assertion on macOS (the RTT-truncation platform), or
- Increase the number of attempts and/or accept `RoundtripTime >= 0` on platforms known to truncate sub-ms RTT, keeping the strict `> 0` assertion only where the raw-socket path preserves sub-ms timing.
## Sample console log
https://helixr1107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-c2d1c1347a4940c8a6/System.Net.Ping.Functional.Tests/3/console.ff3a2daa.log?helixlogtype=result
## Known issue error message
Fill the error message using [known issues](https://github.com/dotnet/arcade/blob/main/Documentation/Projects/Build%20Analysis/KnownIssues.md).
```json
{
"ErrorMessage": "Expected at least one TtlExpired reply with non-zero RoundtripTime across 3 attempts",
"BuildRetry": false,
"ExcludeConsoleLog": false
}
```
### Known issue validation
**Build: :mag_right:** https://dev.azure.com/dnceng-public/public/_build/results?buildId=1546596
**Error message validated:** `[Expected at least one TtlExpired reply with non-zero RoundtripTime across 3 attempts`]
**Result validation:** :white_check_mark: Known issue matched with the provided build.
**Validation performed at:** 8/11/2026 9:19:16 AM UTC
### Report
#### Summary
|24-Hour Hit Count|7-Day Hit Count|1-Month Count|
|---|---|---|
|0|0|0|
Contributor guide
Research direction
Start in src/libraries/System.Net.Ping/tests/FunctionalTests/PingTest.cs at PingTest.SendPingWithLowTtl_RoundtripTimeIsNonZero and read the existing macOS RTT comment. Run the test on the OSX.26.Arm64.Open queue and verify the change handles sub-millisecond RoundtripTime values without weakening the expected TTL-expired response checks or allowing the outerloop failure to recur.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, macos
- Domain
- ci-cd, networking, testing-qa
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100