oxidecomputer / oxidecomputer/usdt
stapsdt, monormoprhisation, and linker --gc-sections.
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 145
- Forks
- 15
- Avg merge
- 14m
- Merged PRs (30d)
- 6
Description
I've spent a few days trying to debug why I'm getting weird behaviour with usdt on linux.
What I'm observing is many stapsdt notes that end up linked into my ELF file have the probe address unresolved. The conditions for this bug to appear are quite hard to minimise for:
- It requires a probe to be part of a monomorphised function
- The monomorphised function needs to be instantiated across multiple codegen units
- Using the lld or mold linker
By eliminating any of these the behaviour would not manifest. After some time chatting with peers. I was able to find that the bug disappears with RUSTFLAGS=-Clink-args=-Wl,--no-gc-sections. This leads me to conclude the bug is the following:
- A function is monomorphised across several CGU.
- This monomorphised function contains the stapsdt note.
- The linker deduplicates these functions when applying GC
- The linker does not remove any dangling notes.
I've found 2 ways to resolve the issue in my testing. But I don't understand whether there's any other consequences of the code they produce. They both relate to the following line
Retained
The first fix is to mark the note as "retained". This ensures that the functions don't get de-duplicated and thus the notes do not dangle. This is not ideal as this causes binary bloat.
.pushsection .note.stapsdt, "R", "note"
Comdat
The second fix I don't fully understand but it works in all my testing and I can't find any problems with the resulting ELF. It uses the following change:
.pushsection .note.stapsdt, "G", "note", ".note.stapsdt.{prov}.{probe}", comdat
cc @aapoalas
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at usdt-impl/src/stapsdt.rs line 156 and reproduce the unresolved probe addresses with monomorphised functions, multiple codegen units, lld or mold, and --gc-sections. Compare the retained and Comdat section variants, then verify that the chosen behavior prevents dangling stapsdt notes without unintended ELF or binary-size consequences.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- linux, rust
- Domain
- build-system, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100