llvm / llvm/llvm-project

[llvm-dwarfutil] The parallel linker ignores --no-garbage-collection, and never garbage collects a partial unit

Open
#219,652 0 comments 0 reactions 0 assignees View on GitHub
llvm-dwarfutil
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

## Summary

The parallel linker keeps a different set of DIEs than the classic linker does, in both garbage collection modes, through two paths in the same keep walk.

1. With `--no-garbage-collection` it still collects garbage. `--no-garbage-collection` turns off the only mechanism `llvm-dwarfutil` documents for discarding debug info, so the tool should fall back to its stated basic behaviour of making a semantic copy of the input. It drops DIEs that the classic linker keeps from the same input, under the same options.
2. With garbage collection enabled it does not collect a partial unit at all. Change the root tag of a unit to `DW_TAG_partial_unit` and nothing else, and every DIE in that unit survives, including the ones the option exists to remove.

The two errors partially cancel, which is what makes this hard to see from one run. Note, that a partial unit under `--no-garbage-collection` is the only one of the four combinations that produces the right output, and it produces it by accident: those DIEs survive because the unit was never collected, not because the option was honoured.

| linker | garbage collection | `DW_TAG_compile_unit` root | `DW_TAG_partial_unit` root |
|---|---|---|---|
| classic | on | dropped | dropped |
| classic | `--no-garbage-collection` | kept | kept |
| parallel | on | dropped | **kept** |
| parallel | `--no-garbage-collection` | **dropped** | **kept** |

The classic linker is correct in all four cells and does not vary with the root tag.

## Reproducer

Three ordinary DWARFv5 compilation units. `U02` contains a namespace `ns` holding a structure `S`, and nothing refers to either. `--build-accelerator=DWARF` is present only because `--no-garbage-collection` on its own copies the file without running the linker at all.

### `--no-garbage-collection` is ignored

Classic keeps them:

```console
$ llvm-dwarfutil --no-garbage-collection --build-accelerator=DWARF cu-only.o cu-only.classic
$ llvm-dwarfdump --debug-info cu-only.classic
0x00000044: DW_TAG_compile_unit
DW_AT_name ("U02")
0x0000005d: DW_TAG_subprogram
DW_AT_name ("foo2")
0x0000006f: DW_TAG_namespace
DW_AT_name ("ns")
0x00000071: DW_TAG_structure_type
DW_AT_name ("S")
```

Parallel drops them:

```console
$ llvm-dwarfutil --linker parallel --no-garbage-collection --build-accelerator=DWARF cu-only.o cu-only.parallel
$ llvm-dwarfdump --debug-info cu-only.parallel
0x00000044: DW_TAG_compile_unit
DW_AT_name ("U02")
0x0000005d: DW_TAG_subprogram
DW_AT_name ("foo2")
0x0000006f: DW_TAG_base_type
DW_AT_name ("int")
0x00000071: NULL
```

Observe, that `int` outlives the `struct S` whose only member referenced it.

### A partial unit is never collected

With garbage collection left enabled, a compile unit root gets the expected result:

```console
$ llvm-dwarfutil --linker parallel cu-only.o cu-only.gc
$ llvm-dwarfdump --debug-info cu-only.gc
0x00000044: DW_TAG_compile_unit
DW_AT_name ("U02")
0x0000005d: DW_TAG_subprogram
DW_AT_name ("foo2")
0x0000006f: DW_TAG_base_type
DW_AT_name ("int")
0x00000071: NULL
```

Changing `U02`'s root to `DW_TAG_partial_unit` and its unit type to `DW_UT_partial`, and nothing else, keeps the whole unit:

```console
$ llvm-dwarfutil --linker parallel pu-only.o pu-only.gc
$ llvm-dwarfdump --debug-info pu-only.gc
0x00000044: DW_TAG_partial_unit
DW_AT_name ("U02")
0x0000005d: DW_TAG_subprogram
DW_AT_name ("foo2")
0x0000006f: DW_TAG_namespace
DW_AT_name ("ns")
0x00000071: DW_TAG_structure_type
DW_AT_name ("S")
0x00000074: DW_TAG_member
DW_AT_name ("m1")
0x0000007d: DW_TAG_base_type
DW_AT_name ("int")
```

The classic linker drops `ns` and `S` for both roots here, as it should. `DW_TAG_partial_unit` is the root tag `dwz` emits, so this is the ordinary shape of a Fedora, RHEL or Debian debuginfo package: garbage collection is silently a no-op on the units that carry most of the types.

## Analysis

### `--no-garbage-collection` never reaches the keep walk

`--no-garbage-collection` reaches the linker as `setUpdateIndexTablesOnly(true)`:

https://github.com/llvm/llvm-project/blob/d34be0ff2eb68d0e486016d8278695fce7d80c3f/llvm/tools/llvm-dwarfutil/DebugInfoLinker.cpp#L374

```cpp
DebugInfoLinker->setUpdateIndexTablesOnly(!Options.DoGarbageCollection);
```

The classic linker replaces its whole keep walk with an unconditional mark in that mode:

https://github.com/llvm/llvm-project/blob/d34be0ff2eb68d0e486016d8278695fce7d80c3f/llvm/lib/DWARFLinker/Classic/DWARFLinker.cpp#L3186-L3189

```cpp
if (LLVM_UNLIKELY(Options.Update)) {
for (auto &CurrentUnit : OptContext.CompileUnits)
CurrentUnit->markEverythingAsKept();
```

The parallel linker has no equivalent. Liveness analysis runs on the same path whatever the option says:

https://github.com/llvm/llvm-project/blob/d34be0ff2eb68d0e486016d8278695fce7d80c3f/llvm/lib/DWARFLinker/Parallel/DWARFLinkerImpl.cpp#L664-L672

```cpp
case CompileUnit::Stage::Loaded: {
// Mark all the DIEs that need to be present in the generated output.
// If ODR requested, build type names.
if (!CU.resolveDependenciesAndMarkLiveness(InterCUProcessingStarted,
HasNewInterconnectedCUs)) {
```

The option's entire effect on marking is to withhold one flag:

https://github.com/llvm/llvm-project/blob/d34be0ff2eb68d0e486016d8278695fce7d80c3f/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp#L171-L172

```cpp
if (!isClangModule() && !getGlobalData().getOptions().UpdateIndexTablesOnly)
ChildInfo.setTrackLiveness();
```

`TrackLiveness` is read in exactly three places, all inside `isLiveVariableEntry` and `isLiveSubprogramEntry`, so all it achieves is making subprograms, variables and labels unconditionally live. `collectRootsToKeep` still selects roots by tag, so a DIE it does not recognize as a root — every namespace, every type, and everything beneath them — is never marked and is dropped exactly as before. `DW_TAG_base_type` survives only because of its own explicit "Always keep base types" case.

### The keep walk treats a partial unit root as an entity rather than as a scope

https://github.com/llvm/llvm-project/blob/d34be0ff2eb68d0e486016d8278695fce7d80c3f/llvm/lib/DWARFLinker/Parallel/DependencyTracker.cpp#L109-L119

```cpp
static bool isNamespaceLikeEntry(const DWARFDebugInfoEntry *Entry) {
switch (Entry->getTag()) {
case dwarf::DW_TAG_compile_unit:
case dwarf::DW_TAG_module:
case dwarf::DW_TAG_namespace:
return true;

default:
return false;
}
}
```

`DW_TAG_partial_unit` is absent, and so are `DW_TAG_type_unit` and `DW_TAG_skeleton_unit`. `markParentsAsKeepingChildren` walks up from a marked DIE and consults that predicate on every ancestor, including the unit root:

https://github.com/llvm/llvm-project/blob/d34be0ff2eb68d0e486016d8278695fce7d80c3f/llvm/lib/DWARFLinker/Parallel/DependencyTracker.cpp#L493-L505

```cpp
if (!ArePlainParentsDone && NeedKeepPlainChildren) {
if (ParentInfo.getKeepPlainChildren())
ArePlainParentsDone = true;
else {
bool AddToWorklist = !isAlreadyMarked(
ParentInfo, CompileUnit::DieOutputPlacement::PlainDwarf);
ParentInfo.setKeepPlainChildren();
if (AddToWorklist && !isNamespaceLikeEntry(ParentEntry)) {
addActionToRootEntriesWorkList(
LiveRootWorklistActionTy::MarkLiveChildrenRec,
UnitEntryPairTy{Entry.CU, ParentEntry}, std::nullopt);
}
}
}
```

A compile unit root takes the namespace-like branch and schedules nothing, so only the DIEs that were individually marked survive. A partial unit root falls through, and the first live child anywhere in the unit — `foo2` in the reproducer — schedules `MarkLiveChildrenRec` on the root itself, which recursively marks every DIE the unit contains. The predicate is asking whether a DIE is a scope container or an entity whose children are its constituent parts, and a unit root is a scope container whatever tag it carries.

## Existing tests

`llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/empty-CU.test` pins the first behaviour and says so:

```
RUN: dsymutil --linker parallel --update -f %t.o -o - | llvm-dwarfdump -v - -debug-info | FileCheck %s

CHECK: .debug_info contents:
CHECK-NOT: DW_TAG_compile_unit
```

while its classic twin at `llvm/test/tools/dsymutil/X86/empty-CU.test` expects the root to be present, and closes with `## FIXME: Support --linker parallel`.

`llvm/test/tools/llvm-dwarfutil/ELF/X86/dwarf5-partial-unit-types-only.test`, which #219615 adds, pins the second behaviour in four `CHECK-PU` lines and states it outright:

> The first unit is emptied either way, since its only child moved to the type unit, but the two roots are then treated differently: a compile unit with nothing left is dropped whole, while a partial unit root is still emitted. That difference is the linker's existing behaviour for a unit with no live contents, not something this fix introduces, so it is pinned rather than made uniform.

That paragraph describes this defect, observed there and deliberately left alone. A fix for this issue has to make those two roots uniform and rewrite those lines, so it depends on #219615.

## Documentation

`llvm/docs/CommandGuide/llvm-dwarfutil.rst` describes the tool's basic contract:

> In basic usage, it makes a semantic copy of the input to the output. If any options are specified, the output may be modified along the way, e.g. by removing unused debug info.

and scopes the option to tombstoned entries specifically:

> `--garbage-collection`
>
> Removes pieces of debug information related to discarded sections. When the linker does section garbage collection the abandoned debug info is left behind. Such abandoned debug info references address ranges using tombstone values. Thus, when this option is specified, the tool removes debug info which is marked with the tombstone value.

> `--no-garbage-collection`
>
> Disable `--garbage-collection`.

`ns` and `S` carry no address ranges at all, so they can neither be related to a discarded section under any reading, nor be exempt from collection when it is requested.

## Expected behavior

With `--no-garbage-collection` the parallel linker keeps every DIE the input contains, as the classic linker does. With garbage collection enabled it collects a partial unit on the same terms as a compile unit. Neither result varies with the root tag, so both linkers produce the same set of DIEs in both modes.

## Versions

The code sites are unchanged on `main` as of d34be0ff2eb68d0e486016d8278695fce7d80c3f, and a build of that tree reproduces every cell of the table above.

This report was produced with AI assistance; I have reviewed it and am accountable for it.

Contributor guide

Open the contributing guide

Research direction

Start with the parallel linker paths in llvm/lib/DWARFLinker/Parallel/DWARFLinkerImpl.cpp, DWARFLinkerCompileUnit.cpp, and DependencyTracker.cpp, then compare them with the classic implementation. Run the existing empty-CU.test and dwarf5-partial-unit-types-only.test cases, updating coverage for both garbage-collection modes; done means parallel and classic retain the same DIEs for compile and partial units. The partial-unit test depends on #219615.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
56/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.