llvm / llvm/llvm-project

[clang][DependencyScanning] Implementing a Streaming Style By-name Dependency Scanning API

Open
#206,818 3 comments 0 reactions 1 assignee Claimed by @qiongsiwu View on GitHub
clang
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

This issue tracks the work to implement a streaming style by-name dependency scanning API.

### Background and motivation
We introduced `CompilerInstanceWithContext` a while back (https://github.com/llvm/llvm-project/commit/be0aa7b6c72bdb162f1f3fe251e469927118963e) to improve the performance of by-name dependency scanning by sharing the compiler's state across multiple by-name lookups. This introduced two issues.
1. The shared compiler instance's state has to be explicitly managed. Users need to call explicit APIs to initialize the shared compiler instance and make sure it is cleaned up properly.
2. The dependency scanner uses two seemingly different code paths to perform dependency scanning. The by-name scanning goes through `CompilerInstanceWithContext`, and the translation unit scanning goes through `DependencyScanningAction`. Although these two code paths call many common functions, they are still different.

### Solution
The solution partitions the responsibilities across the dependency scanning objects in the following way:
1. `DependencyScanningTool` - the main outward facing API that is the entry point to dependency scanning. It can take driver commands or cc1 commands. It manages a `DependencyScanningWorker` instance.
2. `DependencyScanningWorker` - manages the set up of the main scanning engine, the `CompilerInstaneWithContext` instance, and dispatches the scanning tasks (e.g. names to scan) to the main engine.
3. `CompilerInstanceWithContext`- the main scanning engine that takes a cc1 command and performs the dependency scanning.

This partition of responsibility allows a clean separation between these three classes. `CompilerInstanceWithContext` only shows up inside `DependencyScanningWorker`'s methods as an implementation detail. The partition also simplifies the dependency graph between them. These three classes now form a dependency chain (`DependencyScanningTool` depends on `DependencyScanningWorker` depends on `CompilerInstanceWithContext` *), and only `DependencyScanningTool` dependents on `Driver`.

This potentially enables us to implement a streaming style TU dependency scanning API as well. That is out of the scope here, but it is something we can do easier with the changes above. A second thing that could be enabled is better diagnostics.

Update on July 22: this is now implemented as a sequence of stacked PRs starting with https://github.com/llvm/llvm-project/pull/211402.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.