llvm / llvm/llvm-project

TableGen files rebuild unnecessarily even when nothing changed

Open
#167,864 0 comments 0 reactions 0 assignees View on GitHub
cmake
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

I'm working on a fork of LLVM and when I build using `ninja clang && ninja llc && ninja opt` with no changes, TableGen files still try to rebuild (though they finish immediately), probably because some dependency files are missing. This happens every time I run the build, even when nothing has changed.

A few things solved the issue for me, but they are mostly a workaround:

1. **Adding the input filename to the depfile manually**:
I added a line in `createDependencyFile()` in `llvm/lib/TableGen/Main.cpp` to explicitly write the input filename to the depfile:
```cpp
DepOut.os() << ' ' << InputFilename;
```

2. **Forcing the depfile mode to be disabled**: I changed the condition in `TableGen.cmake` from:
```cmake
if(CMAKE_GENERATOR MATCHES "Ninja" AND cmp0116_state STREQUAL NEW ...
```
to:
```cmake
if(FALSE AND CMAKE_GENERATOR MATCHES "Ninja" AND cmp0116_state STREQUAL NEW ...
```
This forces it to use the fallback mode (the `else` branch) which uses explicit file globbing instead of depfiles. When I do this, the rebuild issue goes away completely.

## What I think might be wrong

When the depfile mode is active (the `if` branch), CMake relies entirely on the `.d` depfile to know what files the TableGen output depends on. However, I suspect the depfile might not be including the main input `.td` file itself - only the files that are included via `include` statements.

## Environment

- LLVM: Current main branch (post commit ab8b8c1)
- CMake: 3.20+ (which defaults CMP0116 to NEW)
- Generator: Ninja
- Platform: Linux

## Questions

Is this a known issue? Am I misunderstanding how the depfile should work? Should the main input file be included in the depfile, or is there another mechanism that's supposed to handle it?

Contributor guide

Open the contributing guide

Research direction

Reproduce with `ninja clang && ninja llc && ninja opt` on Linux using Ninja and CMake 3.20+; start in `llvm/lib/TableGen/Main.cpp` at `createDependencyFile()` and inspect the depfile generated by the TableGen rules in `TableGen.cmake`. Compare depfile mode with the fallback globbing mode, and consider the issue done when unchanged builds no longer attempt to rebuild TableGen outputs.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp
Domain
build-system, compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.