dotnet-pgo: support supplementary perf sample files for SPGO and call graph
- Dominant language
- C#
- Stars
- 18.3k
- Forks
- 5.6k
- PR merge metrics
- PR metrics pending
Description
## Summary
`dotnet-pgo create-mibc` builds its call graph (for Pettis-Hansen method layout) and SPGO block counts from `SampledProfileTraceData` events in ETW/EventPipe traces. However, when using Linux perfcollect traces, the perf CPU samples in `perf.data` are not converted to `SampledProfileTraceData` events in the .etlx - they remain as raw IP addresses in `perf.data.txt`.
This means:
1. **Pettis-Hansen gets no call graph data** - `CallWeights` is empty, method layout is silently skipped
2. **SPGO block attribution misses perf samples** - only ETW-originated samples are attributed
## Proposal
Add two optional parameters to `dotnet-pgo create-mibc`:
### `--supplementary-samples `
A text file with one hex instruction pointer per line (leaf IP of each CPU sample). These are fed to `SampleCorrelator.AttributeSamplesToIP()` for SPGO basic block attribution.
### `--supplementary-callgraph `
A text file with `callee_hex_ip caller_hex_ip` pairs per line (adjacent frames from CPU sampling callstacks). These are resolved against the `MethodMemoryMap` to populate `callGraph` and `exclusiveSamples` for Pettis-Hansen method layout.
Both files can be trivially extracted from perfcollect's `perf.data.txt` by parsing the callstack frames.
## Context
We're building a PGO pipeline for [Nethermind](https://github.com/NethermindEth/nethermind) (Ethereum execution client) using perfcollect for CPU sampling on Linux. The pipeline works end-to-end but requires patching dotnet-pgo at build time to read these supplementary files.
Currently we extract ~9.6M perf samples from perfcollect traces, of which ~1.9M are attributed to managed code (the rest is kernel/native). This produces SPGO block counts for 1,350 methods across 32 assemblies. The call graph data would enable Pettis-Hansen method layout which is currently a no-op due to missing `CallWeights`.
Related issues:
- #125883 - SPGO fails with perfcollect traces (CTF mapping issues)
- #125896 - FlowSmoothing crash in SPGO
- microsoft/perfview#2392 - Missing CTF event mappings in TraceEvent
## Alternative
Ideally, `CreateFromLinuxEventSources` in TraceEvent would properly correlate perf.data samples with CLR method maps and produce `SampledProfileTraceData` events in the .etlx. This would make the supplementary files unnecessary. However, the current implementation doesn't do this correlation for perfcollect traces, and fixing it in TraceEvent is a larger change.
Contributor guide
Assessment
This issue has not been assessed yet.