bazelbuild / bazelbuild/bazel

C++ toolchain callbacks

Open
#17,237 12 comments 1 reaction 0 assignees View on GitHub
help wanted P2 team-Rules-CPP type: feature request
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

## Description
Add the option to create a Starlark method tied to the C++ toolchain callable before actions

There are many feature requests that involve adding code to the Bazel C++ rules in order to pipe the variables necessary for very specific use cases:

- a new output by the compilation or linking actions not yet contemplated by the rules

- flags that have to be generated dynamically and therefore cannot go in copts or linkopts but for which there isn’t an existing build variable

It would scale better if the C++ rules code base didn’t have to add explicit support for each of these use cases.

## Work
The community will drive adding support for this feature, it first requires discussion in a design doc and reviewing existing issues labeled team-rules-Cpp in bazelbuild/bazel to make sure that we consider every use case that could benefit from this. Once the design is approved, the author can proceed with implementation.

Issues that could benefit (important to find more to justify the work here):
1. Custom build variable with short filename: https://github.com/bazelbuild/bazel/issues/15924

1. AOSP issue for linking which can benefit from this: https://github.com/bazelbuild/bazel/issues/17277

## Possible implementation
(This is an idea of how I imagine it roughly, final implementation should be guided by the design.)

### Placement
Currently the C++ rules are structured as follows:
#### Compilation

1. Rule outer later (cc_library, cc_binary, etc..)

1. CcCompilationHelper.java (we will start rewriting to Starlark this quarter)

1. CppCompileActionBuilder

1. CppCompileAction

#### Linking

1. Rule outer later (cc_library, cc_binary, etc..)

1. CcLinkingHelper

1. CppLinkActoinBuilder

1. CppLinkAction

The toolchain callback should probably go between 2 and 3, in other words, it should be called once for every CppCompileAction and every CppLinkAction created. This allows adding the custom output per action.

### Possible callback signature for CppCompileAction
```
def toolchain_callback(ctx, source): # is cc_toolchain and feature config needed here?
# Logic specific to each implementation goes here
# …..
# Output which would then be read from the C++ rule’s implementation and passed
# when constructing the CppCompile action
return {
“additional_inputs” : new_inputs,
“additional_outputs” : new_outputs,
“extensions” : extensions, # Build variable extensions for the crosstool
}
```

The extensions would be a simple dictionary like the one passed [here](https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/rules/cpp/CcModule.java;drc=1623f7eb7850b6e92010d5f76607c0cb019cb159;l=2249).

For https://github.com/bazelbuild/bazel/issues/15924, the flags can be created in a loop one by one based on the source passed.

There are plans for Q1 and Q2 to starlarkify CcCompilationHelper but meanwhile the part of the C++ rules logic that would invoke the callback is still in Java despite the toolchain callback being in Starlark. Calling Starlark code from Java is possible via Starlark.call (see [here](https://cs.opensource.google/bazel/bazel/+/master:src/test/java/net/starlark/java/eval/ScriptTest.java;drc=7c0bdc2c1c428a9bcd74f47e05d9c475ff43535d;l=117), [here](https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/analysis/starlark/TemplateDict.java;drc=87834594eff2a312b84832f382dc1f8844b1e8a3;l=131) and [here](https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/analysis/RuleContext.java;drc=38349753009edaddb1d0ef104035ab01423aefec;l=1161)).

The compilation callback would be stored in the CcToolchainProvider and called before creating every CppCompilation action. Similarly for the linking callback. The callback can be passed as an argument [here](https://cs.opensource.google/bazel/bazel/+/master:tools/cpp/unix_cc_toolchain_config.bzl;drc=abae5ca3e8142f93cf0c2597e3410ed955c4dd59;l=1380). This feature request only makes sense for projects that are using Bazel's C++ rules and want to add customization on top, a project with custom rules doesn't need this feature.

The suggestions above can be modified completely after experimenting with real use cases. After the design discussion and experimenting contributors may also conclude that this isn't really needed and there are better ways to achieve the same thing.

Contributor guide

Open the contributing guide

Research direction

Start by writing the requested design document and reviewing team-rules-Cpp issues, especially #15924 and #17277, to establish the use cases. Then inspect CcCompilationHelper, CcLinkingHelper, CppCompileActionBuilder, CppLinkActionBuilder, CcToolchainProvider, and the cited Starlark.call examples. Done means an approved design and an implementation plan that addresses the relevant compilation and linking callbacks.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, java
Domain
build-system, compilers
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.