bazel-contrib / bazel-contrib/rules_python

exec tools exec_interpreter broken on RBE

未关闭
#2,703 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
type: bug
主要语言
Starlark
星标
688
派生
721
平均合并
15 小时 7 分钟
30 天内合并 PR
76

描述

As found in https://github.com/bazel-contrib/rules_python/pull/2657, the `exec_interpreter` attribute of the exec tools toolchain doesn't work with RBE.

What happens is the `current_interpreter_executable()` helper rule does `declare_file(); symlink()`, which puts the file at e.g. `python/private/python3`. Locally, this creates a symlink and when run, Python follows the symlink back to the actual location of the installed runtime ("python home"); this works. On RBE, however, it creates copy of the file. Python then can't find its way back to "python home" and fails to find e.g. shared libs, stdlib, etc.

The intent of the `exec_interpreter` field was to make it easy to run an interpreter in a build action by having it support `DefaultInfo.files_to_run`, which lets it be passed directly to `ctx.actions.run(executable=...)`. Without a FilesToRun provider, the interpreter and its files have to be manually passed in. Additionally, because actions can't directly accept runfiles, it means the usual way to express the runfiles necessary for an executable (DefaultInfo.default_runfiles) can't be used.

-----

While looking into solutions, a promising option looked something like:
* Have py_runtime() return DefaultInfo with executable set.
* In order to do this, re-declare the interpreter file as a file. This, apparently, works, even if it "clobbers" the original file. This is necessary because DefaultInfo.executable requires the file to be produced by the same rule.
* In py_runtime_pair, have it put the DefaultInfo that py_runtime produced on the ToolchainInfo it returned
* Alternative: create a new PyRuntimeInfo with a replaced interpreter field

Consumers are then able to do e.g.

```
exe = ctx.toolchains["exec_tools"].interpreter # DefaultInfo.files_to_run object
```

Unfortunately, I didn't see a decent way to preserve the "exec_interpreter is a target that you can pass to ctx.actions.run" behavior. _Maybe_ this would be possible if the underlying py_runtime target (which would have DefaultInfo.executable) could be more directly plumbed though. I'm not sure that's possible, though.

贡献指南

打开贡献指南

调研方向

首先跟踪 current_interpreter_executable() 以及通过 py_runtime() 和 py_runtime_pair() 的 exec_interpreter 路径。检查 DefaultInfo.files_to_run、DefaultInfo.executable 和 runfiles 如何暴露给 ctx.actions.run(executable=...)。完成标准是 exec interpreter 能够在 RBE 下工作,同时保留预期的直接 executable 用法。

由索引模型根据 Issue 内容生成。

评估

技术栈
python
领域
build-system, infrastructure
Issue 类型
缺陷
难度
5/5
预计耗时
一周以上
活跃度
停滞
描述清晰度
基本清楚
新手友好度
30/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。