bazel-contrib / bazel-contrib/rules_python

compile_pip_requirements update and all_whl_requirements seem like a circular dependency

オープン
#2,836 コメント 2 件 リアクション 0 件 担当者 0 名 GitHub で見る
type: documentation type: pip
主要言語
Starlark
スター
688
フォーク
721
平均マージ
15時間 7分
マージ済み PR(30日)
76

説明

# 🐞 bug report

### Affected Rule

The issue is caused by the rule:
`compile_pip_requirements`
`all_whl_requirements`

### Is this a regression?

Have not tested previous versions.

### Description

A clear and concise description of the problem...
When trying to switch the index url I noticed that requirements.txt was getting populated with a package that didnt exist in the requirements.txt file when running the `compile_pip_requirements.update` rule. This should have failed from the beginning as the registry for the index_url was bogus. Then with further poking around I ran it again and this time it failed. There are 2 concerns:
1. How is the requirements.txt being filled in with a package when overriding the index url to one that doesnt contain the package?
2. The rule `all_whl_requirements` seems to be loading before the update command causing it to pass with the empty requirements.txt and then fail the second time when running the same command since the package doesnt actually exist in the new index registry.

## 🔬 Minimal Reproduction

`MODULE.bazel`:
```
bazel_dep(name = "rules_python", version = "1.3.0")

python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(
configure_coverage_tool = True,
ignore_root_user_error = True,
python_version = "3.11",
)
use_repo(python, "python_3_11")

pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
experimental_index_url = "https://us-python.pkg.dev/null/simple",
hub_name = "pypi",
python_version = "3.11",
requirements_lock = "//:requirements.txt",
)
use_repo(pip, "pypi")
```

`requirements.in`:
```
itsdangerous
```

`requirements.txt` is empty to start.

`BUILD.bazel`:
```
load("@pypi//:requirements.bzl", "all_whl_requirements")
load("@rules_python//python:pip.bzl", "compile_pip_requirements")

compile_pip_requirements(
name = "requirements",
src = "requirements.in",
extra_args = [
"--allow-unsafe",
],
requirements_txt = "requirements.txt",
)
```

To produce error:
1. Execute the command: `bazel run //:requirements.update`
2. Execute the command again: `bazel run //:requirements.update`

## 🔥 Exception or Error

```
$ bazel run //:requirements.update
INFO: Analyzed target //:requirements.update (131 packages loaded, 5471 targets configured).
INFO: Found 1 target...
Target //:requirements.update up-to-date:
bazel-bin/requirements.update
INFO: Elapsed time: 6.987s, Critical Path: 0.59s
INFO: 5 processes: 5 internal.
INFO: Build completed successfully, 5 total actions
INFO: Running command line: bazel-bin/requirements.update '--src=_main/requirements.in' _main/requirements.txt //:requirements.update '--resolver=backtracking' --allow-unsafe --generate-hashes --allow-unsafe
Updating requirements.txt
WARNING: --strip-extras is becoming the default in version 8.0.0. To silence this warning, either use --strip-extras to opt into the new default or use --no-strip-extras to retain the existing behavior.
#
# This file is autogenerated by pip-compile with Python 3.11
# by the following command:
#
# bazel run //:requirements.update
#
itsdangerous==2.2.0 \
--hash=sha256:c6242fc49e35958c8b15141343aa660db5fc54d4f13a1db01a3f5891b98700ef \
--hash=sha256:e0050c0b7da1eea53ffaf149c0cfbb5c6e2e2b69c4bef22c81fa6eb73e5f6173
# via -r requirements.in
$ bazel run //:requirements.update
WARNING: Download from https://us-python.pkg.dev/null/simple/itsdangerous/ failed: class java.io.FileNotFoundException GET returned 404 Not Found
ERROR: ~/.cache/bazel/_bazel_user/f7d37797d74a062b8f39bc84d0ebc222/external/rules_python~/python/private/pypi/simpleapi_download.bzl:130:14: Traceback (most recent call last):
File "~/.cache/bazel/_bazel_user/f7d37797d74a062b8f39bc84d0ebc222/external/rules_python~/python/private/pypi/extension.bzl", line 615, column 25, in _pip_impl
mods = parse_modules(module_ctx)
File "~/.cache/bazel/_bazel_user/f7d37797d74a062b8f39bc84d0ebc222/external/rules_python~/python/private/pypi/extension.bzl", line 490, column 36, in parse_modules
out = _create_whl_repos(
File "~/.cache/bazel/_bazel_user/f7d37797d74a062b8f39bc84d0ebc222/external/rules_python~/python/private/pypi/extension.bzl", line 155, column 50, in _create_whl_repos
requirements_by_platform = parse_requirements(
File "~/.cache/bazel/_bazel_user/f7d37797d74a062b8f39bc84d0ebc222/external/rules_python~/python/private/pypi/parse_requirements.bzl", line 180, column 36, in parse_requirements
index_urls = get_index_urls(
File "~/.cache/bazel/_bazel_user/f7d37797d74a062b8f39bc84d0ebc222/external/rules_python~/python/private/pypi/extension.bzl", line 470, column 79, in lambda
get_index_urls = lambda ctx, distributions: simpleapi_download(
File "~/.cache/bazel/_bazel_user/f7d37797d74a062b8f39bc84d0ebc222/external/rules_python~/python/private/pypi/simpleapi_download.bzl", line 130, column 14, in simpleapi_download
_fail("Failed to download metadata for {} for from urls: {}".format(
Error in fail: Failed to download metadata for ["itsdangerous"] for from urls: ["https://us-python.pkg.dev/null/simple"]
ERROR: error evaluating module extension pip in @@rules_python~//python/extensions:pip.bzl
INFO: Elapsed time: 0.521s
INFO: 0 processes.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target
FAILED:
Fetching module extension pip in @@rules_python~//python/extensions:pip.bzl; starting
```

## 🌍 Your Environment

**Operating System:**

  

Linux

**Output of `bazel version`:**

  

$ bazel version
Bazelisk version: development
Build label: 7.4.0
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Tue Oct 22 17:24:25 2024 (1729617865)
Build timestamp: 1729617865
Build timestamp as int: 1729617865

**Rules_python version:**

  

1.3.0

**Anything else relevant?**

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

示されている MODULE.bazel、BUILD.bazel、requirements.in、空の requirements.txt を使って、issue にある 2 つの `bazel run //:requirements.update` コマンドを再現します。次に、報告されたスタックトレースの周辺にある `python/private/pypi/extension.bzl`、`parse_requirements.bzl`、`simpleapi_download.bzl` を読みます。完了の条件は、無効なインデックス URL が一貫して処理され、最初の実行でパッケージが生成されず、循環依存も作成されないことです。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
python
領域
build-system
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
停滞
明瞭さ
おおむね明確
初心者へのやさしさ
35/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。