llvm / llvm/llvm-project

[llvm-dwarfutil] ODR deduplication is silently skipped for types under a DW_TAG_partial_unit root

Open
#219,393 0 comments 0 reactions 0 assignees View on GitHub
llvm-dwarfutil
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

## Summary

ODR type deduplication is silently skipped for every type under a `DW_TAG_partial_unit` root, in the classic and parallel DWARFLinker backends alike. The output is valid DWARF, but each partial unit keeps its own copy of every type, which defeats `--odr-deduplication` on exactly the input where duplicate type definitions are most abundant.

`DW_TAG_partial_unit` is what `dwz` leaves behind, and `dwz` is a standard step in distribution debuginfo packaging (Fedora and RHEL run it from `find-debuginfo`, Debian from `dh_dwz`), so this is not a hypothetical shape.

## Reproducer

The input below holds two partial units, each defining `struct S` with two members and each carrying `DW_AT_language`.

```yaml
--- !ELF
FileHeader:
Class: ELFCLASS64
Data: ELFDATA2LSB
Type: ET_REL
Machine: EM_X86_64
Sections:
- Name: .text
Type: SHT_PROGBITS
Flags: [ SHF_ALLOC, SHF_EXECINSTR ]
Address: 0x1130
Size: 0x30
- Name: .debug_addr
Type: SHT_PROGBITS
Flags: [ ]
Content: "0c000000050008003011000000000000"
DWARF:
debug_abbrev:
- ID: 0
Table:
- Tag: DW_TAG_partial_unit
Children: DW_CHILDREN_yes
Attributes:
- Attribute: DW_AT_producer
Form: DW_FORM_string
- Attribute: DW_AT_name
Form: DW_FORM_string
- Attribute: DW_AT_language
Form: DW_FORM_data2
- Attribute: DW_AT_low_pc
Form: DW_FORM_addr
- Attribute: DW_AT_high_pc
Form: DW_FORM_data8
- Attribute: DW_AT_addr_base
Form: DW_FORM_sec_offset
- Tag: DW_TAG_subprogram
Children: DW_CHILDREN_no
Attributes:
- Attribute: DW_AT_name
Form: DW_FORM_string
- Attribute: DW_AT_low_pc
Form: DW_FORM_addr
- Attribute: DW_AT_high_pc
Form: DW_FORM_data8
- Attribute: DW_AT_type
Form: DW_FORM_ref4
- Tag: DW_TAG_structure_type
Children: DW_CHILDREN_yes
Attributes:
- Attribute: DW_AT_name
Form: DW_FORM_string
- Attribute: DW_AT_byte_size
Form: DW_FORM_data1
- Tag: DW_TAG_member
Children: DW_CHILDREN_no
Attributes:
- Attribute: DW_AT_name
Form: DW_FORM_string
- Attribute: DW_AT_type
Form: DW_FORM_ref4
- Attribute: DW_AT_data_member_location
Form: DW_FORM_data1
- Tag: DW_TAG_base_type
Children: DW_CHILDREN_no
Attributes:
- Attribute: DW_AT_name
Form: DW_FORM_string
debug_info:
- Version: 5
UnitType: DW_UT_partial
AbbrevTableID: 0
AbbrOffset: 0x0
Entries:
- AbbrCode: 1
Values:
- CStr: by_hand
- CStr: PU1
- Value: 0x04
- Value: 0x1130
- Value: 0x30
- Value: 0x8
- AbbrCode: 2
Values:
- CStr: foo1
- Value: 0x1130
- Value: 0x10
- Value: 0x63
- AbbrCode: 2
Values:
- CStr: foo2
- Value: 0x1150
- Value: 0x10
- Value: 0x63
- AbbrCode: 3
Values:
- CStr: S
- Value: 0x8
- AbbrCode: 4
Values:
- CStr: m1
- Value: 0x7a
- Value: 0x0
- AbbrCode: 4
Values:
- CStr: m2
- Value: 0x7a
- Value: 0x4
- AbbrCode: 0
- AbbrCode: 5
Values:
- CStr: int
- AbbrCode: 0
- Version: 5
UnitType: DW_UT_partial
AbbrevTableID: 0
AbbrOffset: 0x0
Entries:
- AbbrCode: 1
Values:
- CStr: by_hand
- CStr: PU2
- Value: 0x04
- Value: 0x1130
- Value: 0x30
- Value: 0x8
- AbbrCode: 2
Values:
- CStr: bar1
- Value: 0x1130
- Value: 0x10
- Value: 0x63
- AbbrCode: 2
Values:
- CStr: bar2
- Value: 0x1150
- Value: 0x10
- Value: 0x63
- AbbrCode: 3
Values:
- CStr: S
- Value: 0x8
- AbbrCode: 4
Values:
- CStr: m1
- Value: 0x7a
- Value: 0x0
- AbbrCode: 4
Values:
- CStr: m2
- Value: 0x7a
- Value: 0x4
- AbbrCode: 0
- AbbrCode: 5
Values:
- CStr: int
- AbbrCode: 0
...
```

```console
$ yaml2obj partial.yaml -o partial.o
$ llvm-dwarfutil --linker parallel --garbage-collection --odr-deduplication partial.o partial.out
$ llvm-dwarfdump --debug-info partial.out | grep -c DW_TAG_structure_type
2
```

Substituting `DW_TAG_compile_unit`/`DW_UT_compile` for `DW_TAG_partial_unit`/`DW_UT_partial` in the same input gives:

```console
$ llvm-dwarfdump --debug-info compile.out | grep -c DW_TAG_structure_type
1
```

With compile unit roots the output holds three units: an artificial type unit carrying the single deduplicated `S`, plus the two compile units. With partial unit roots no type unit is created at all. The classic backend behaves the same way, one copy against two.

## Analysis

Firstly, the classic backend stops building decl contexts at anything that is not a `DW_TAG_compile_unit`, so nothing inside a partial unit ever acquires a context to unique against:

https://github.com/llvm/llvm-project/blob/a7831dcce98893f3514f889a827f0d38da46621a/llvm/lib/DWARFLinker/Classic/DWARFLinkerDeclContext.cpp#L69-L77

```cpp
switch (Tag) {
default:
// By default stop gathering child contexts.
return PointerIntPair(nullptr);
...
case dwarf::DW_TAG_compile_unit:
return PointerIntPair(&Context);
```

Secondly, the parallel backend arrives at the same outcome by a different route. `isNamespaceLikeEntry` lists `DW_TAG_compile_unit`, `DW_TAG_module` and `DW_TAG_namespace`, but not `DW_TAG_partial_unit`:

https://github.com/llvm/llvm-project/blob/a7831dcce98893f3514f889a827f0d38da46621a/llvm/lib/DWARFLinker/Parallel/DependencyTracker.cpp#L109-L119

`getRootForSpecifiedEntry` walks up the parent chain until `isNamespaceLikeEntry` returns true:

https://github.com/llvm/llvm-project/blob/a7831dcce98893f3514f889a827f0d38da46621a/llvm/lib/DWARFLinker/Parallel/DependencyTracker.cpp#L949-L957

`getRootForSpecifiedEntry` terminates at the type itself under a compile unit root. Under a partial unit root it climbs past the unit root instead and returns the root DIE as the type root, and `getTypeDeduplicationCandidate` returns `std::nullopt` for a unit root, so nothing is ever registered as a deduplication candidate:

https://github.com/llvm/llvm-project/blob/a7831dcce98893f3514f889a827f0d38da46621a/llvm/lib/DWARFLinker/Parallel/SyntheticTypeNameBuilder.cpp#L167-L175

## Implementation notes

`cloneDIEandCloneAttributes` gates type DIE cloning on the root tag rather than on the DIE being the unit root:

https://github.com/llvm/llvm-project/blob/a7831dcce98893f3514f889a827f0d38da46621a/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp#L1420-L1423

`DependencyTracker` sets `setKeepTypeChildren()` on the parent unconditionally while propagating type liveness upward, so a `DW_TAG_partial_unit` root satisfies `needToPlaceInTypeTable()` where a compile unit root is excluded by that tag test. Reaching it calls `createTypeDIEandCloneAttributes()` on DIE index 0, which asserts because no type entry is ever assigned to a unit root:

https://github.com/llvm/llvm-project/blob/a7831dcce98893f3514f889a827f0d38da46621a/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp#L1605-L1609

The assert on DIE index 0 is unreachable today only because the deduplication described above never engages. A fix that restores deduplication without also changing that gate turns the assert into a crash.

## Expected behavior

Types under a `DW_TAG_partial_unit` root participate in ODR deduplication on the same terms as types under a `DW_TAG_compile_unit` root.

## Versions

Reproduced at a7831dcce98893f3514f889a827f0d38da46621a on `main`, x86_64 Linux, on both the classic (default) and `--linker parallel` backends. `dsymutil` shares the same code and enables ODR deduplication by default.

This report was produced with AI assistance; I have reviewed it and am accountable for it.

Contributor guide

Open the contributing guide

Research direction

Start by running the supplied partial.yaml reproducer with yaml2obj, llvm-dwarfutil, and llvm-dwarfdump. Read DWARFLinker/Classic/DWARFLinkerDeclContext.cpp, Parallel/DependencyTracker.cpp, Parallel/SyntheticTypeNameBuilder.cpp, and Parallel/DWARFLinkerCompileUnit.cpp, focusing on the cited entry points and tag checks. Done means both classic and parallel backends deduplicate the partial-unit types without asserting, producing one structure type as with compile units.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.