bazelbuild / bazelbuild/bazel-skylib

Add helper rule/provider for working with files inside tree artifacts

Open
#300 3 comments 0 reactions 0 assignees View on GitHub
P3 type: feature request
Dominant language
Starlark
Stars
444
Forks
202
PR merge metrics
No merged PRs in 30d

Description

TreeArtifacts have a lot of shortcomings. One of them is that it's impossible for an action to produce labels both for a directory output (`ctx.actions.declare_directory`) and *also* a label for a file within that directory. (some parts of Bazel fail to strictly disallow this, but at least the RBE protocol fails when you do this.)

In rules_nodejs we are working on providing dependencies as directories. This is because in the npm ecosystem there's no equivalent of a .jar file, so our dependencies are exploded and easily reach 10k input files to an action, making sandbox setup time a significant portion of action execution time.

However there's no accepted way across the Bazel ecosystem to continue referencing one file within a directory. One use case where you need this in many languages is the "main". Take https://docs.bazel.build/versions/master/be/python.html#py_binary.main for example - if you wanted to supply a directory (TreeArtifact) to the `srcs` of a `py_binary` you'd have no way to say which file within that directory should be executed.

We'll probably invent our own solution just for rules_nodejs (https://github.com/bazelbuild/rules_nodejs/pull/2579) but we'd rather upstream this concept into skylib so that rulesets can be more interoperable when they need to pass each other a construct of type `{Label of TreeArtifact, string path within that label}`

Contributor guide

Open the contributing guide

Research direction

Start with the py_binary.main entry point and the rules_nodejs pull request #2579 to understand the TreeArtifact limitation and existing proposed approach. The work is done when skylib provides an interoperable helper rule or provider carrying a TreeArtifact label together with a path to a file inside it.

Written by the indexing model from the issue text.

Assessment

Domain
build-system
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.