bazel-contrib / bazel-contrib/rules_python

Hermetic toolchains incur a significant overhead to running py_binary and py_test

Offen
#1,653 13 Kommentare 8 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Starlark
Sterne
688
Forks
721
Ø Merge
15 Std. 7 Min.
Gemergte PRs (30 T.)
76

Beschreibung

# 🐞 bug report

### Affected Rule

`py_binary` and `py_test`

### Is this a regression?

No

### Description

Hermetic toolchains incur a significant overhead to executing Python targets due to the large amount of files which are added to the runfiles tree.
You find an analysis of the issue here: https://github.com/martis42/show_hermetic_python_overhead

Bazel caching of course negates much of this issue in daily working.
Still, whenever a test has to rerun we pay this additional overhead.

I don't know if there is a "perfect" solution possible given how Bazel sandboxing works.
However, it would be great if rues_python would add a note in its documentation for hermetic toolchains that adding `--nolegacy_external_runfiles` to the `.bazelrc` file is recommended.

Maybe the `rules_python` maintainer can even convince the Bazel maintainer to prioritize working on flipping `--legacy_external_runfiles` in Bazel 8 ?

## 🔬 Minimal Reproduction

You see reproduce this with this small example workspace https://github.com/martis42/show_hermetic_python_overhead

## 🔥 Exception or Error

NA

## 🌍 Your Environment

**Operating System:**

  

Linux Mint 21.1

**Output of `bazel version`:**

  

Bazelisk version: v1.16.0
Build label: 7.0.0
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Mon Dec 11 16:51:49 2023 (1702313509)
Build timestamp: 1702313509
Build timestamp as int: 1702313509

**Rules_python version:**

  

0.27.0

**Anything else relevant?**

https://github.com/bazelbuild/rules_python/issues/1624 might be related.
It is however maybe Windows specific.

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Start with the linked show_hermetic_python_overhead workspace and compare py_binary and py_test execution with hermetic toolchains. Read the hermetic-toolchain documentation and the .bazelrc setting --nolegacy_external_runfiles; done means the overhead and recommendation are documented clearly, or the limitation is explained.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
python
Bereich
build-system, performance
Issue-Typ
Bug
Schwierigkeit
3/5
Geschätzter Aufwand
1-2 Tage
Aktivitätsstatus
Veraltet
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
45/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.