aboutcode-org / aboutcode-org/python-inspector

Problem with resolution

Offen
#101 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
23
Forks
28
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

There are a couple issues:
**1. this file is not a valid requirements file IMHO and we do not return any error:**
I save this in re.txt and ran:
` python-inspector --json - -p 3.10 -o linux --version -r re.txt`

```
numpy>=1.0
scipy
xobjects
xpart
xdeps

[tests]
cpymad
PyHEADTAIL
pytest
```
This was copied by mistake as is from the setup.py of xtrack-0.23.2.tar.gz from https://pypi.org/project/xtrack/#files

**2. trying to inspect it yields this error which is not right becuase this setup.py is perfectly parsable:**
```
>>> resolver_api(requirement_files=["re.txt"], python_version="310", operating_system="linux")
Traceback (most recent call last):
File "", line 1, in
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/api.py", line 254, in resolve_dependencies
resolution, purls = resolve(
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/api.py", line 313, in resolve
resolved_dependencies, packages = get_resolved_dependencies(
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/api.py", line 351, in get_resolved_dependencies
resolver_results = resolver.resolve(requirements=requirements, max_rounds=max_rounds)
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 521, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 402, in resolve
failure_causes = self._attempt_to_pin_criterion(name)
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 238, in _attempt_to_pin_criterion
criteria = self._get_updated_criteria(candidate)
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 228, in _get_updated_criteria
for requirement in self._p.get_dependencies(candidate=candidate):
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/resolution.py", line 600, in get_dependencies
return list(self._iter_dependencies(candidate))
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/resolution.py", line 591, in _iter_dependencies
for r in self.get_requirements_for_package(purl=purl, candidate=candidate):
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/resolution.py", line 470, in get_requirements_for_package_from_pypi_simple
yield from get_requirements_from_python_manifest(
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/resolution.py", line 304, in get_requirements_from_python_manifest
raise Exception(
Exception: Unable to collect setup.py dependencies securely: .cache/thirdparty/extracted_sdists/PyHEADTAIL-1.16.0/PyHEADTAIL-1.16.0/setup.py

```

**3. Trying to call the API with the setup.py is not right:**
See https://files.pythonhosted.org/packages/7c/dd/80d8ec272179941bf6cc181446127da194eb82b7138549a6939583c4c318/PyHEADTAIL-1.16.0.tar.gz (which should have been in the graph above) returns an imcomplete deps tree (or rather nothing)
```
>>> resolver_api(setup_py_file=".cache/thirdparty/extracted_sdists/PyHEADTAIL-1.16.0/PyHEADTAIL-1.16.0/setup.py", python_version="310", operating_system="linux")
Resolution(resolution=[], packages=[], files=[{'type': 'file', 'path': '.cache/thirdparty/extracted_sdists/PyHEADTAIL-1.16.0/PyHEADTAIL-1.16.0/setup.py', 'package_data': [{'type': 'pypi', 'namespace': None, 'name': 'PyHEADTAIL', 'version': '1.16.0', 'qualifiers': {}, 'subpath': None, 'primary_language': 'Python', 'description': 'CERN PyHEADTAIL numerical n-body simulation code for simulating macro-particle beam dynamics with collective effects.', 'release_date': None, 'parties': [{'type': 'person', 'role': 'author', 'name': 'Kevin Li', 'email': 'Kevin.Shing.Bruce.Li@cern.ch', 'url': None}, {'type': 'person', 'role': 'maintainer', 'name': 'Adrian Oeftiger', 'email': 'Adrian.Oeftiger@cern.ch', 'url': None}], 'keywords': [], 'homepage_url': 'https://github.com/PyCOMPLETE/PyHEADTAIL', 'download_url': None, 'size': None, 'sha1': None, 'md5': None, 'sha256': None, 'sha512': None, 'bug_tracking_url': None, 'code_view_url': None, 'vcs_url': None, 'copyright': None, 'license_expression': None, 'declared_license': {}, 'notice_text': None, 'source_packages': [], 'file_references': [], 'extra_data': {}, 'dependencies': [{'purl': 'pkg:pypi/h5py', 'extracted_requirement': 'h5py', 'scope': 'setup', 'is_runtime': True, 'is_optional': False, 'is_resolved': False, 'resolved_package': {}, 'extra_data': {}}, {'purl': 'pkg:pypi/numpy', 'extracted_requirement': 'numpy', 'scope': 'setup', 'is_runtime': True, 'is_optional': False, 'is_resolved': False, 'resolved_package': {}, 'extra_data': {}}, {'purl': 'pkg:pypi/scipy', 'extracted_requirement': 'scipy', 'scope': 'setup', 'is_runtime': True, 'is_optional': False, 'is_resolved': False, 'resolved_package': {}, 'extra_data': {}}, {'purl': 'pkg:pypi/cython', 'extracted_requirement': 'cython', 'scope': 'setup', 'is_runtime': True, 'is_optional': False, 'is_resolved': False, 'resolved_package': {}, 'extra_data': {}}], 'repository_homepage_url': 'https://pypi.org/project/PyHEADTAIL', 'repository_download_url': 'https://pypi.org/packages/source/P/PyHEADTAIL/PyHEADTAIL-1.16.0.tar.gz', 'api_data_url': 'https://pypi.org/pypi/PyHEADTAIL/1.16.0/json', 'datasource_id': 'pypi_setup_py', 'purl': 'pkg:pypi/pyheadtail@1.16.0'}]}])
```

**4. Scanning that seame setup.py with SCTK works without a glitch with these deps**
```
dependencies:
- purl: pkg:pypi/h5py
extracted_requirement: h5py
scope: setup
is_runtime: yes
is_optional: no
is_resolved: no
resolved_package: {}
extra_data: {}
- purl: pkg:pypi/numpy
extracted_requirement: numpy
scope: setup
is_runtime: yes
is_optional: no
is_resolved: no
resolved_package: {}
extra_data: {}
- purl: pkg:pypi/scipy
extracted_requirement: scipy
scope: setup
is_runtime: yes
is_optional: no
is_resolved: no
resolved_package: {}
extra_data: {}
- purl: pkg:pypi/cython
extracted_requirement: cython
scope: setup
is_runtime: yes
is_optional: no
is_resolved: no
resolved_package: {}
extra_data: {}

```

**5. running this does not have the extra requires:**
`python-inspector --spec "xtrack==0.23.3" --json - -p 3.10 -o linux`

```
"resolved_dependencies_graph": [
{
"package": "pkg:pypi/cffi@1.15.1",
"dependencies": [
"pkg:pypi/pycparser@2.21"
]
},
{
"package": "pkg:pypi/lark@1.1.4",
"dependencies": []
},
{
"package": "pkg:pypi/numpy@1.23.5",
"dependencies": []
},
{
"package": "pkg:pypi/pycparser@2.21",
"dependencies": []
},
{
"package": "pkg:pypi/scipy@1.9.3",
"dependencies": [
"pkg:pypi/numpy@1.23.5"
]
},
{
"package": "pkg:pypi/xdeps@0.0.10",
"dependencies": [
"pkg:pypi/lark@1.1.4"
]
},
{
"package": "pkg:pypi/xobjects@0.1.27",
"dependencies": [
"pkg:pypi/cffi@1.15.1",
"pkg:pypi/numpy@1.23.5"
]
},
{
"package": "pkg:pypi/xpart@0.10.0",
"dependencies": [
"pkg:pypi/numpy@1.23.5",
"pkg:pypi/scipy@1.9.3",
"pkg:pypi/xobjects@0.1.27"
]
},
{
"package": "pkg:pypi/xtrack@0.23.3",
"dependencies": [
"pkg:pypi/numpy@1.23.5",
"pkg:pypi/scipy@1.9.3",
"pkg:pypi/xdeps@0.0.10",
"pkg:pypi/xobjects@0.1.27",
"pkg:pypi/xpart@0.10.0"
]
}
]
```

**6. we should have a way to return partial resolutions and we should not fail silently**

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.