[llvm-dwarfutil] ODR deduplication is silently skipped for types under a DW_TAG_partial_unit root
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
## Summary
ODR type deduplication is silently skipped for every type under a `DW_TAG_partial_unit` root, in the classic and parallel DWARFLinker backends alike. The output is valid DWARF, but each partial unit keeps its own copy of every type, which defeats `--odr-deduplication` on exactly the input where duplicate type definitions are most abundant.
`DW_TAG_partial_unit` is what `dwz` leaves behind, and `dwz` is a standard step in distribution debuginfo packaging (Fedora and RHEL run it from `find-debuginfo`, Debian from `dh_dwz`), so this is not a hypothetical shape.
## Reproducer
The input below holds two partial units, each defining `struct S` with two members and each carrying `DW_AT_language`.
```yaml
--- !ELF
FileHeader:
Class: ELFCLASS64
Data: ELFDATA2LSB
Type: ET_REL
Machine: EM_X86_64
Sections:
- Name: .text
Type: SHT_PROGBITS
Flags: [ SHF_ALLOC, SHF_EXECINSTR ]
Address: 0x1130
Size: 0x30
- Name: .debug_addr
Type: SHT_PROGBITS
Flags: [ ]
Content: "0c000000050008003011000000000000"
DWARF:
debug_abbrev:
- ID: 0
Table:
- Tag: DW_TAG_partial_unit
Children: DW_CHILDREN_yes
Attributes:
- Attribute: DW_AT_producer
Form: DW_FORM_string
- Attribute: DW_AT_name
Form: DW_FORM_string
- Attribute: DW_AT_language
Form: DW_FORM_data2
- Attribute: DW_AT_low_pc
Form: DW_FORM_addr
- Attribute: DW_AT_high_pc
Form: DW_FORM_data8
- Attribute: DW_AT_addr_base
Form: DW_FORM_sec_offset
- Tag: DW_TAG_subprogram
Children: DW_CHILDREN_no
Attributes:
- Attribute: DW_AT_name
Form: DW_FORM_string
- Attribute: DW_AT_low_pc
Form: DW_FORM_addr
- Attribute: DW_AT_high_pc
Form: DW_FORM_data8
- Attribute: DW_AT_type
Form: DW_FORM_ref4
- Tag: DW_TAG_structure_type
Children: DW_CHILDREN_yes
Attributes:
- Attribute: DW_AT_name
Form: DW_FORM_string
- Attribute: DW_AT_byte_size
Form: DW_FORM_data1
- Tag: DW_TAG_member
Children: DW_CHILDREN_no
Attributes:
- Attribute: DW_AT_name
Form: DW_FORM_string
- Attribute: DW_AT_type
Form: DW_FORM_ref4
- Attribute: DW_AT_data_member_location
Form: DW_FORM_data1
- Tag: DW_TAG_base_type
Children: DW_CHILDREN_no
Attributes:
- Attribute: DW_AT_name
Form: DW_FORM_string
debug_info:
- Version: 5
UnitType: DW_UT_partial
AbbrevTableID: 0
AbbrOffset: 0x0
Entries:
- AbbrCode: 1
Values:
- CStr: by_hand
- CStr: PU1
- Value: 0x04
- Value: 0x1130
- Value: 0x30
- Value: 0x8
- AbbrCode: 2
Values:
- CStr: foo1
- Value: 0x1130
- Value: 0x10
- Value: 0x63
- AbbrCode: 2
Values:
- CStr: foo2
- Value: 0x1150
- Value: 0x10
- Value: 0x63
- AbbrCode: 3
Values:
- CStr: S
- Value: 0x8
- AbbrCode: 4
Values:
- CStr: m1
- Value: 0x7a
- Value: 0x0
- AbbrCode: 4
Values:
- CStr: m2
- Value: 0x7a
- Value: 0x4
- AbbrCode: 0
- AbbrCode: 5
Values:
- CStr: int
- AbbrCode: 0
- Version: 5
UnitType: DW_UT_partial
AbbrevTableID: 0
AbbrOffset: 0x0
Entries:
- AbbrCode: 1
Values:
- CStr: by_hand
- CStr: PU2
- Value: 0x04
- Value: 0x1130
- Value: 0x30
- Value: 0x8
- AbbrCode: 2
Values:
- CStr: bar1
- Value: 0x1130
- Value: 0x10
- Value: 0x63
- AbbrCode: 2
Values:
- CStr: bar2
- Value: 0x1150
- Value: 0x10
- Value: 0x63
- AbbrCode: 3
Values:
- CStr: S
- Value: 0x8
- AbbrCode: 4
Values:
- CStr: m1
- Value: 0x7a
- Value: 0x0
- AbbrCode: 4
Values:
- CStr: m2
- Value: 0x7a
- Value: 0x4
- AbbrCode: 0
- AbbrCode: 5
Values:
- CStr: int
- AbbrCode: 0
...
```
```console
$ yaml2obj partial.yaml -o partial.o
$ llvm-dwarfutil --linker parallel --garbage-collection --odr-deduplication partial.o partial.out
$ llvm-dwarfdump --debug-info partial.out | grep -c DW_TAG_structure_type
2
```
Substituting `DW_TAG_compile_unit`/`DW_UT_compile` for `DW_TAG_partial_unit`/`DW_UT_partial` in the same input gives:
```console
$ llvm-dwarfdump --debug-info compile.out | grep -c DW_TAG_structure_type
1
```
With compile unit roots the output holds three units: an artificial type unit carrying the single deduplicated `S`, plus the two compile units. With partial unit roots no type unit is created at all. The classic backend behaves the same way, one copy against two.
## Analysis
Firstly, the classic backend stops building decl contexts at anything that is not a `DW_TAG_compile_unit`, so nothing inside a partial unit ever acquires a context to unique against:
https://github.com/llvm/llvm-project/blob/a7831dcce98893f3514f889a827f0d38da46621a/llvm/lib/DWARFLinker/Classic/DWARFLinkerDeclContext.cpp#L69-L77
```cpp
switch (Tag) {
default:
// By default stop gathering child contexts.
return PointerIntPair(nullptr);
...
case dwarf::DW_TAG_compile_unit:
return PointerIntPair(&Context);
```
Secondly, the parallel backend arrives at the same outcome by a different route. `isNamespaceLikeEntry` lists `DW_TAG_compile_unit`, `DW_TAG_module` and `DW_TAG_namespace`, but not `DW_TAG_partial_unit`:
https://github.com/llvm/llvm-project/blob/a7831dcce98893f3514f889a827f0d38da46621a/llvm/lib/DWARFLinker/Parallel/DependencyTracker.cpp#L109-L119
`getRootForSpecifiedEntry` walks up the parent chain until `isNamespaceLikeEntry` returns true:
https://github.com/llvm/llvm-project/blob/a7831dcce98893f3514f889a827f0d38da46621a/llvm/lib/DWARFLinker/Parallel/DependencyTracker.cpp#L949-L957
`getRootForSpecifiedEntry` terminates at the type itself under a compile unit root. Under a partial unit root it climbs past the unit root instead and returns the root DIE as the type root, and `getTypeDeduplicationCandidate` returns `std::nullopt` for a unit root, so nothing is ever registered as a deduplication candidate:
https://github.com/llvm/llvm-project/blob/a7831dcce98893f3514f889a827f0d38da46621a/llvm/lib/DWARFLinker/Parallel/SyntheticTypeNameBuilder.cpp#L167-L175
## Implementation notes
`cloneDIEandCloneAttributes` gates type DIE cloning on the root tag rather than on the DIE being the unit root:
https://github.com/llvm/llvm-project/blob/a7831dcce98893f3514f889a827f0d38da46621a/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp#L1420-L1423
`DependencyTracker` sets `setKeepTypeChildren()` on the parent unconditionally while propagating type liveness upward, so a `DW_TAG_partial_unit` root satisfies `needToPlaceInTypeTable()` where a compile unit root is excluded by that tag test. Reaching it calls `createTypeDIEandCloneAttributes()` on DIE index 0, which asserts because no type entry is ever assigned to a unit root:
https://github.com/llvm/llvm-project/blob/a7831dcce98893f3514f889a827f0d38da46621a/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp#L1605-L1609
The assert on DIE index 0 is unreachable today only because the deduplication described above never engages. A fix that restores deduplication without also changing that gate turns the assert into a crash.
## Expected behavior
Types under a `DW_TAG_partial_unit` root participate in ODR deduplication on the same terms as types under a `DW_TAG_compile_unit` root.
## Versions
Reproduced at a7831dcce98893f3514f889a827f0d38da46621a on `main`, x86_64 Linux, on both the classic (default) and `--linker parallel` backends. `dsymutil` shares the same code and enables ODR deduplication by default.
This report was produced with AI assistance; I have reviewed it and am accountable for it.
Contributor guide
Research direction
Start by running the supplied partial.yaml reproducer with yaml2obj, llvm-dwarfutil, and llvm-dwarfdump. Read DWARFLinker/Classic/DWARFLinkerDeclContext.cpp, Parallel/DependencyTracker.cpp, Parallel/SyntheticTypeNameBuilder.cpp, and Parallel/DWARFLinkerCompileUnit.cpp, focusing on the cited entry points and tag checks. Done means both classic and parallel backends deduplicate the partial-unit types without asserting, producing one structure type as with compile units.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100