llvm / llvm/llvm-project

lld/COFF rejects COMDAT selection type 0 although MSVC accepts it (breaks linking of real-world vendor libraries)

Open Beginner friendly
#211,317 1 comment 0 reactions 0 assignees View on GitHub
lld:COFF rejects-valid
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

Summary
ld.lld: rejects COFF object files that contain COMDAT sections with selection type = 0, reporting:
```
unknown comdat type 0 for in
```
However, Microsoft’s link.exe accepts COMDAT selection type 0 and links these objects successfully.
This strict validation in LLVM breaks linking of real-world vendor libraries (e.g., HASP/Aladdin/SafeNet), which rely on MSVC’s permissive behavior.

LLVM’s stated goal for COFF is MSVC compatibility.
Therefore, rejecting COMDAT type 0 is a compatibility bug.

Environment
LLVM version: 21.x (also reproducible on 18–20)

Platform: Windows / MinGW / llvm-mingw cross toolchains

Linker: ld.lld

Input: COFF .obj inside vendor .lib (HASP Windows x64)

Steps to reproduce
Obtain a COFF object file containing COMDAT sections with selection type 0
(e.g., libhasp_windows_x64_demo.lib from the HASP SDK).

Attempt to link it with ld.lld:

`ld.lld foo.obj libhasp_windows_x64_demo.lib`
Observe the fatal error:

`unknown comdat type 0 for in `
Link the same object with MSVC’s link.exe:

`link.exe foo.obj libhasp_windows_x64_demo.lib`
MSVC links successfully without warnings.

Expected behavior (MSVC-compatible)
MSVC accepts COMDAT selection type 0 and treats it as IMAGE_COMDAT_SELECT_LARGEST.
This behavior has been stable across MSVC versions for decades.

LLVM should follow MSVC’s behavior for COFF compatibility.

Actual behavior (LLVM)
ld.lld rejects the object and aborts:

In lld/COFF/InputFiles.cpp:
```cpp
if (def->Selection < IMAGE_COMDAT_SELECT_NODUPLICATES ||
def->Selection > IMAGE_COMDAT_SELECT_LARGEST) {
Fatal(ctx) << "unknown comdat type "
<< std::to_string((int)def->Selection) << " for " << getName()
<< " in " << toString(this);
}
```
This strict validation is incompatible with MSVC.

Several commercial SDKs (including HASP/SafeNet/Aladdin) ship COFF objects with:

- COMDAT sections on .text and .rdata
- selection type = 0
- missing or malformed symbol auxiliary records
- MSVC-compatible but not spec-compliant COFF

These libraries link fine with MSVC but fail with LLVM.

This prevents users from adopting LLVM toolchains for Windows builds

Proposed fix
Match MSVC behavior by treating invalid/undefined COMDAT selection types as IMAGE_COMDAT_SELECT_LARGEST instead of rejecting them.

Suggested patch:

```cpp
// Match MSVC behavior: treat invalid COMDAT selection type as LARGEST
if (def->Selection < IMAGE_COMDAT_SELECT_NODUPLICATES ||
def->Selection > IMAGE_COMDAT_SELECT_LARGEST) {
def->Selection = IMAGE_COMDAT_SELECT_LARGEST;
}
```

Contributor guide

Open the contributing guide

Research direction

Start in lld/COFF/InputFiles.cpp at the COMDAT selection validation that reports “unknown comdat type 0.” Reproduce with foo.obj and libhasp_windows_x64_demo.lib, compare against link.exe, and verify that ld.lld accepts selection type 0 as IMAGE_COMDAT_SELECT_LARGEST and links the vendor library successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.