bazel-contrib / bazel-contrib/rules_python

exec tools exec_interpreter broken on RBE

Abierto
#2,703 0 comentarios 0 reacciones 0 asignados Ver en GitHub
type: bug
Lenguaje dominante
Starlark
Estrellas
688
Forks
721
Merge medio
15 h 7 min
PR fusionados (30 d)
76

Descripción

As found in https://github.com/bazel-contrib/rules_python/pull/2657, the `exec_interpreter` attribute of the exec tools toolchain doesn't work with RBE.

What happens is the `current_interpreter_executable()` helper rule does `declare_file(); symlink()`, which puts the file at e.g. `python/private/python3`. Locally, this creates a symlink and when run, Python follows the symlink back to the actual location of the installed runtime ("python home"); this works. On RBE, however, it creates copy of the file. Python then can't find its way back to "python home" and fails to find e.g. shared libs, stdlib, etc.

The intent of the `exec_interpreter` field was to make it easy to run an interpreter in a build action by having it support `DefaultInfo.files_to_run`, which lets it be passed directly to `ctx.actions.run(executable=...)`. Without a FilesToRun provider, the interpreter and its files have to be manually passed in. Additionally, because actions can't directly accept runfiles, it means the usual way to express the runfiles necessary for an executable (DefaultInfo.default_runfiles) can't be used.

-----

While looking into solutions, a promising option looked something like:
* Have py_runtime() return DefaultInfo with executable set.
* In order to do this, re-declare the interpreter file as a file. This, apparently, works, even if it "clobbers" the original file. This is necessary because DefaultInfo.executable requires the file to be produced by the same rule.
* In py_runtime_pair, have it put the DefaultInfo that py_runtime produced on the ToolchainInfo it returned
* Alternative: create a new PyRuntimeInfo with a replaced interpreter field

Consumers are then able to do e.g.

```
exe = ctx.toolchains["exec_tools"].interpreter # DefaultInfo.files_to_run object
```

Unfortunately, I didn't see a decent way to preserve the "exec_interpreter is a target that you can pass to ctx.actions.run" behavior. _Maybe_ this would be possible if the underlying py_runtime target (which would have DefaultInfo.executable) could be more directly plumbed though. I'm not sure that's possible, though.

Guía de contribución

Abrir la guía de contribución

Línea de trabajo

Empieza siguiendo current_interpreter_executable() y la ruta de exec_interpreter a través de py_runtime() y py_runtime_pair(). Comprueba cómo se exponen DefaultInfo.files_to_run, DefaultInfo.executable y runfiles a ctx.actions.run(executable=...). Se considera terminado cuando un exec interpreter funciona bajo RBE y se conserva el uso directo previsto del ejecutable.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
python
Área
build-system, infrastructure
Tipo de issue
Error
Dificultad
5/5
Tiempo estimado
Más de una semana
Estado de actividad
Estancado
Claridad
Bastante claro
Aptitud para principiantes
30/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.