Disk polling: investigate tail latency under concurrent writes
- Dominant language
- Rust
- Stars
- 4.9k
- Forks
- 432
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 173
Description
Part of #4174, which records the compared revisions, workload and statistical method.
Investigate the unresolved tail behavior under concurrent writes. Start with archived timelines; collect more diagnostics only when a recurring delay or an explicit tail requirement makes the decision useful.
| Case | Change in run p99 | Pointwise 95% interval |
| --- | ---: | --- |
| Explicit offset | −0.02% | [−2.79%, +2.88%] |
| Explicit offset with writes | +3.40% | [−0.62%, +7.63%] |
| Next | −0.84% | [−2.37%, +0.69%] |
| Consumer group | +0.86% | [−0.89%, +2.65%] |
| Resident | −2.33% | [−6.29%, +1.23%] |
Positive values mean higher p99. Estimates are geometric means of paired ratios of run p99 values, not pooled request percentiles. They use the same pair bootstrap described in #4174. The interval with writes includes a small improvement and a meaningful increase, so it neither confirms a tail regression nor resolves a 3% margin. Mean elapsed time is a separate metric, reported in the parent.
The preceding campaign with the same binaries reported Next p99 +6.79% and group p99 +6.06%; those increases were not reproduced. Keep the campaigns separate. The rerun followed reported high Docker memory consumption and a guest restart, but memory pressure was not independently controlled and continuously measured. No container CPU throttling or memory limit events were recorded; those checks cannot exclude host pressure or establish a memory cause.
1. Inspect the [published raw timelines and resource records](https://github.com/diegomrsantos/iggy/releases/tag/pr-4119-final-review-evidence-2026-09-13), including run order and valid slow operations. Look for clustering around a stage or resource condition.
2. Reuse the parent's diagnostics and timestamps from #4177. Add sparse stage timestamps, scheduler waits or queue observations only where needed to locate the delay. CPU profiles alone cannot locate waiting.
3. State the latency boundary, load model and worthwhile effect before fresh confirmation. Follow the parent's fixed budget and arrival model rules, including scheduling delays, errors and timeouts.
Tracing can create the tail being studied. Preserve valid slow runs and stop according to the declared budget. Repeating measurements on the same VM cannot remove systematic environment bias or establish physical device latency. A change should address a supported mechanism, not merely a difference between campaigns.
Contributor guide
Research direction
Start with the published raw timelines and resource records from the linked release, keeping run order and valid slow operations separate. Reuse the diagnostics and timestamps from #4177, adding sparse stage or queue observations only when needed. Done means stating the latency boundary and load model, identifying a supported mechanism if one exists, and following the parent issue's fixed budget and arrival-model rules.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, rust
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100