bazelbuild / bazelbuild/bazel

Allow macro instance change pruning under lazy expansion of symbolic macros

Open
#26,128 2 comments 0 reactions 0 assignees View on GitHub
team-Loading-API
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 16h
Merged PRs (30d)
72

Description

Suppose we make a change to a package that doesn't change a MacroInstance. For example, maybe we added an unrelated target to the BUILD file, or changed the definition of a parent symbolic macro in a way that doesn't affect the attributes passed to this macro. In that case, under [lazy macro expansion](https://github.com/bazelbuild/bazel/issues/23852), we want to prune on the macro instance's skyvalue so that descendants of the pruned value do not need to be recomputed.

This is trickier than it looks at first glance.

* We will have to remove other unprunable dependency routes between PackagePieceValues - because if they exist, pruning on macro instances won't help. In particular, this implies PackagePiece.ForMacro now cannot have a pointer to PackagePiece.ForBuildFile. We'll have to copy Package.Declarations/Metadata around, and as for the BUILD file's FileTarget, you'd need to get that from PackagePiece.ForBuildFile - a PackagePiece.ForMacro cannot reference it.
* We will have to remove dependency routes between MacroInstanceValues. This means removing MacroInstance's parent pointer - instead, we'd need a parent ID that the caller can use to find the parent macro instance via skyframe.
* We will have to make MacroInstances more prune-friendly by moving their generator location and Starlark instantiation stack out to their owning package piece. We want to ensure e.g. that a new line added near the beginning of a BUILD file doesn't invalidate all macro instances in the package.

Part of https://github.com/bazelbuild/bazel/issues/23852

Contributor guide

Open the contributing guide

Research direction

Start with issue #23852 and trace PackagePiece.ForMacro, PackagePiece.ForBuildFile, MacroInstance, PackagePieceValues, and MacroInstanceValues through the lazy macro expansion implementation. Map the dependency routes and verify that parent, generator-location, and instantiation-stack data can move as described. Done means unchanged macro instances prune their Skyframe descendants without unrelated BUILD-file changes invalidating them.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.