aboutcode-org / aboutcode-org/python-inspector
`packages` is empty when using a non-pypi.org simple index (e.g. Sonatype Nexus)
- 主要言語
- Python
- スター
- 23
- フォーク
- 28
- PR マージ指標
- 30日以内にマージされた PR はありません
説明
Currently the metadata fetch requires the pypi.org-specific JSON API. (See also #260 )
When `PYINSP_INDEX_URL` points at a non-pypi.org simple index (Sonatype Nexus PyPI in my case), `python-inspector` resolves dependencies but the `packages` array in the output is empty - package metadata is never fetched.
For example: `PYINSP_INDEX_URL=https://my-nexus-pypi.com/repository/pypi/simple`:
```shell
root@2ae255f06ad8:/project# python-inspector -p3.13 -rrequirements.txt -olinux --json-pdt /tmp/test.json --verbose
...
retrieve package data from pypi:
retrieved package 'pkg:pypi/cx-oracle@7.3.0'
retrieved package 'pkg:pypi/redis@2.10.6'
done!
root@2ae255f06ad8:/project# jq '.packages[].name' /tmp/test.json
# (no output - packages is [])
```
Note the verbose log claims `retrieved package ...`, yet `packages` is empty - the retrieval silently failed.
Without the index (default pypi.org) it works:
```shell
root@2ae255f06ad8:/project# env -u PYINSP_INDEX_URL python-inspector -p3.13 -rrequirements.txt -olinux --json-pdt /tmp/test.json --verbose
...
root@2ae255f06ad8:/project# jq '.packages[].name' /tmp/test.json
"cx-oracle"
"redis"
```
Root cause is the `get_pypi_data_from_purl` builds the metadata URL from the index base and queries the pypi.org-specific JSON API, then returns `None` on any non-200:
https://github.com/aboutcode-org/python-inspector/blob/a841c7ce82c6087695af7fc558f5ad30fb5b1e27/src/python_inspector/package_data.py#L65-L77
With a Nexus index this resolves to `https://my-nexus-pypi.com/repository/pypi/pypi/redis/2.10.6/json`, which 404s - Nexus's PyPI format implements the simple index (PEP 503), not the `/pypi///json` API - so the function bails out and no `PackageData` is produced.
I initially assumed `--use-pypi-json-api` was the toggle for this. It doesn't help here: in its default (off) position `get_pypi_data_from_purl` still unconditionally queries the JSON API, and in the on position the help states `--index-url` is ignored (i.e. pypi.org), so it can't reach a private index either way.
I'm not sure what the correct approach would be, I feel like `--use-pypi-json-api` option should be fixed that way we download the selected sdist/wheel and parse its `PKG-INFO` for `License` / `Home-page` / `Author` / `Requires-Dist`. The simple index already provides the file list + digests, so the JSON API's `urls` are largely redundant; its unique value is the rich `info` block, which `PKG-INFO` also carries.
While searching for solution I've also stumbled upon `PEP 691` https://packaging.python.org/en/latest/specifications/simple-repository-api/#json-based-simple-api-for-python-package-indexes
Which might be better than parsing of `PKG-INFO` (`pkginfo` package exists though, so it would be easy)
Either way, would like to hear the maintainers thoughts on that. If it something that should be fixed - I would try.
コントリビューションガイド
このリポジトリのコントリビューションガイドは索引されていません
評価
この issue はまだ評価されていません。