aboutcode-org / aboutcode-org/python-inspector

Problem with resolution

Abierto
#101 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
23
Forks
28
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

There are a couple issues:
**1. this file is not a valid requirements file IMHO and we do not return any error:**
I save this in re.txt and ran:
` python-inspector --json - -p 3.10 -o linux --version -r re.txt`

```
numpy>=1.0
scipy
xobjects
xpart
xdeps

[tests]
cpymad
PyHEADTAIL
pytest
```
This was copied by mistake as is from the setup.py of xtrack-0.23.2.tar.gz from https://pypi.org/project/xtrack/#files

**2. trying to inspect it yields this error which is not right becuase this setup.py is perfectly parsable:**
```
>>> resolver_api(requirement_files=["re.txt"], python_version="310", operating_system="linux")
Traceback (most recent call last):
File "", line 1, in
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/api.py", line 254, in resolve_dependencies
resolution, purls = resolve(
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/api.py", line 313, in resolve
resolved_dependencies, packages = get_resolved_dependencies(
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/api.py", line 351, in get_resolved_dependencies
resolver_results = resolver.resolve(requirements=requirements, max_rounds=max_rounds)
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 521, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 402, in resolve
failure_causes = self._attempt_to_pin_criterion(name)
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 238, in _attempt_to_pin_criterion
criteria = self._get_updated_criteria(candidate)
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 228, in _get_updated_criteria
for requirement in self._p.get_dependencies(candidate=candidate):
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/resolution.py", line 600, in get_dependencies
return list(self._iter_dependencies(candidate))
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/resolution.py", line 591, in _iter_dependencies
for r in self.get_requirements_for_package(purl=purl, candidate=candidate):
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/resolution.py", line 470, in get_requirements_for_package_from_pypi_simple
yield from get_requirements_from_python_manifest(
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/resolution.py", line 304, in get_requirements_from_python_manifest
raise Exception(
Exception: Unable to collect setup.py dependencies securely: .cache/thirdparty/extracted_sdists/PyHEADTAIL-1.16.0/PyHEADTAIL-1.16.0/setup.py

```

**3. Trying to call the API with the setup.py is not right:**
See https://files.pythonhosted.org/packages/7c/dd/80d8ec272179941bf6cc181446127da194eb82b7138549a6939583c4c318/PyHEADTAIL-1.16.0.tar.gz (which should have been in the graph above) returns an imcomplete deps tree (or rather nothing)
```
>>> resolver_api(setup_py_file=".cache/thirdparty/extracted_sdists/PyHEADTAIL-1.16.0/PyHEADTAIL-1.16.0/setup.py", python_version="310", operating_system="linux")
Resolution(resolution=[], packages=[], files=[{'type': 'file', 'path': '.cache/thirdparty/extracted_sdists/PyHEADTAIL-1.16.0/PyHEADTAIL-1.16.0/setup.py', 'package_data': [{'type': 'pypi', 'namespace': None, 'name': 'PyHEADTAIL', 'version': '1.16.0', 'qualifiers': {}, 'subpath': None, 'primary_language': 'Python', 'description': 'CERN PyHEADTAIL numerical n-body simulation code for simulating macro-particle beam dynamics with collective effects.', 'release_date': None, 'parties': [{'type': 'person', 'role': 'author', 'name': 'Kevin Li', 'email': 'Kevin.Shing.Bruce.Li@cern.ch', 'url': None}, {'type': 'person', 'role': 'maintainer', 'name': 'Adrian Oeftiger', 'email': 'Adrian.Oeftiger@cern.ch', 'url': None}], 'keywords': [], 'homepage_url': 'https://github.com/PyCOMPLETE/PyHEADTAIL', 'download_url': None, 'size': None, 'sha1': None, 'md5': None, 'sha256': None, 'sha512': None, 'bug_tracking_url': None, 'code_view_url': None, 'vcs_url': None, 'copyright': None, 'license_expression': None, 'declared_license': {}, 'notice_text': None, 'source_packages': [], 'file_references': [], 'extra_data': {}, 'dependencies': [{'purl': 'pkg:pypi/h5py', 'extracted_requirement': 'h5py', 'scope': 'setup', 'is_runtime': True, 'is_optional': False, 'is_resolved': False, 'resolved_package': {}, 'extra_data': {}}, {'purl': 'pkg:pypi/numpy', 'extracted_requirement': 'numpy', 'scope': 'setup', 'is_runtime': True, 'is_optional': False, 'is_resolved': False, 'resolved_package': {}, 'extra_data': {}}, {'purl': 'pkg:pypi/scipy', 'extracted_requirement': 'scipy', 'scope': 'setup', 'is_runtime': True, 'is_optional': False, 'is_resolved': False, 'resolved_package': {}, 'extra_data': {}}, {'purl': 'pkg:pypi/cython', 'extracted_requirement': 'cython', 'scope': 'setup', 'is_runtime': True, 'is_optional': False, 'is_resolved': False, 'resolved_package': {}, 'extra_data': {}}], 'repository_homepage_url': 'https://pypi.org/project/PyHEADTAIL', 'repository_download_url': 'https://pypi.org/packages/source/P/PyHEADTAIL/PyHEADTAIL-1.16.0.tar.gz', 'api_data_url': 'https://pypi.org/pypi/PyHEADTAIL/1.16.0/json', 'datasource_id': 'pypi_setup_py', 'purl': 'pkg:pypi/pyheadtail@1.16.0'}]}])
```

**4. Scanning that seame setup.py with SCTK works without a glitch with these deps**
```
dependencies:
- purl: pkg:pypi/h5py
extracted_requirement: h5py
scope: setup
is_runtime: yes
is_optional: no
is_resolved: no
resolved_package: {}
extra_data: {}
- purl: pkg:pypi/numpy
extracted_requirement: numpy
scope: setup
is_runtime: yes
is_optional: no
is_resolved: no
resolved_package: {}
extra_data: {}
- purl: pkg:pypi/scipy
extracted_requirement: scipy
scope: setup
is_runtime: yes
is_optional: no
is_resolved: no
resolved_package: {}
extra_data: {}
- purl: pkg:pypi/cython
extracted_requirement: cython
scope: setup
is_runtime: yes
is_optional: no
is_resolved: no
resolved_package: {}
extra_data: {}

```

**5. running this does not have the extra requires:**
`python-inspector --spec "xtrack==0.23.3" --json - -p 3.10 -o linux`

```
"resolved_dependencies_graph": [
{
"package": "pkg:pypi/cffi@1.15.1",
"dependencies": [
"pkg:pypi/pycparser@2.21"
]
},
{
"package": "pkg:pypi/lark@1.1.4",
"dependencies": []
},
{
"package": "pkg:pypi/numpy@1.23.5",
"dependencies": []
},
{
"package": "pkg:pypi/pycparser@2.21",
"dependencies": []
},
{
"package": "pkg:pypi/scipy@1.9.3",
"dependencies": [
"pkg:pypi/numpy@1.23.5"
]
},
{
"package": "pkg:pypi/xdeps@0.0.10",
"dependencies": [
"pkg:pypi/lark@1.1.4"
]
},
{
"package": "pkg:pypi/xobjects@0.1.27",
"dependencies": [
"pkg:pypi/cffi@1.15.1",
"pkg:pypi/numpy@1.23.5"
]
},
{
"package": "pkg:pypi/xpart@0.10.0",
"dependencies": [
"pkg:pypi/numpy@1.23.5",
"pkg:pypi/scipy@1.9.3",
"pkg:pypi/xobjects@0.1.27"
]
},
{
"package": "pkg:pypi/xtrack@0.23.3",
"dependencies": [
"pkg:pypi/numpy@1.23.5",
"pkg:pypi/scipy@1.9.3",
"pkg:pypi/xdeps@0.0.10",
"pkg:pypi/xobjects@0.1.27",
"pkg:pypi/xpart@0.10.0"
]
}
]
```

**6. we should have a way to return partial resolutions and we should not fail silently**

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.