aboutcode-org / aboutcode-org/python-inspector

Problem with resolution

Aberta
#101 0 comentários 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Python
Estrelas
23
Forks
28
Métricas de merge de PRs
Nenhum PR com merge em 30d

Descrição

There are a couple issues:
**1. this file is not a valid requirements file IMHO and we do not return any error:**
I save this in re.txt and ran:
` python-inspector --json - -p 3.10 -o linux --version -r re.txt`

```
numpy>=1.0
scipy
xobjects
xpart
xdeps

[tests]
cpymad
PyHEADTAIL
pytest
```
This was copied by mistake as is from the setup.py of xtrack-0.23.2.tar.gz from https://pypi.org/project/xtrack/#files

**2. trying to inspect it yields this error which is not right becuase this setup.py is perfectly parsable:**
```
>>> resolver_api(requirement_files=["re.txt"], python_version="310", operating_system="linux")
Traceback (most recent call last):
File "", line 1, in
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/api.py", line 254, in resolve_dependencies
resolution, purls = resolve(
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/api.py", line 313, in resolve
resolved_dependencies, packages = get_resolved_dependencies(
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/api.py", line 351, in get_resolved_dependencies
resolver_results = resolver.resolve(requirements=requirements, max_rounds=max_rounds)
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 521, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 402, in resolve
failure_causes = self._attempt_to_pin_criterion(name)
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 238, in _attempt_to_pin_criterion
criteria = self._get_updated_criteria(candidate)
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 228, in _get_updated_criteria
for requirement in self._p.get_dependencies(candidate=candidate):
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/resolution.py", line 600, in get_dependencies
return list(self._iter_dependencies(candidate))
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/resolution.py", line 591, in _iter_dependencies
for r in self.get_requirements_for_package(purl=purl, candidate=candidate):
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/resolution.py", line 470, in get_requirements_for_package_from_pypi_simple
yield from get_requirements_from_python_manifest(
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/resolution.py", line 304, in get_requirements_from_python_manifest
raise Exception(
Exception: Unable to collect setup.py dependencies securely: .cache/thirdparty/extracted_sdists/PyHEADTAIL-1.16.0/PyHEADTAIL-1.16.0/setup.py

```

**3. Trying to call the API with the setup.py is not right:**
See https://files.pythonhosted.org/packages/7c/dd/80d8ec272179941bf6cc181446127da194eb82b7138549a6939583c4c318/PyHEADTAIL-1.16.0.tar.gz (which should have been in the graph above) returns an imcomplete deps tree (or rather nothing)
```
>>> resolver_api(setup_py_file=".cache/thirdparty/extracted_sdists/PyHEADTAIL-1.16.0/PyHEADTAIL-1.16.0/setup.py", python_version="310", operating_system="linux")
Resolution(resolution=[], packages=[], files=[{'type': 'file', 'path': '.cache/thirdparty/extracted_sdists/PyHEADTAIL-1.16.0/PyHEADTAIL-1.16.0/setup.py', 'package_data': [{'type': 'pypi', 'namespace': None, 'name': 'PyHEADTAIL', 'version': '1.16.0', 'qualifiers': {}, 'subpath': None, 'primary_language': 'Python', 'description': 'CERN PyHEADTAIL numerical n-body simulation code for simulating macro-particle beam dynamics with collective effects.', 'release_date': None, 'parties': [{'type': 'person', 'role': 'author', 'name': 'Kevin Li', 'email': 'Kevin.Shing.Bruce.Li@cern.ch', 'url': None}, {'type': 'person', 'role': 'maintainer', 'name': 'Adrian Oeftiger', 'email': 'Adrian.Oeftiger@cern.ch', 'url': None}], 'keywords': [], 'homepage_url': 'https://github.com/PyCOMPLETE/PyHEADTAIL', 'download_url': None, 'size': None, 'sha1': None, 'md5': None, 'sha256': None, 'sha512': None, 'bug_tracking_url': None, 'code_view_url': None, 'vcs_url': None, 'copyright': None, 'license_expression': None, 'declared_license': {}, 'notice_text': None, 'source_packages': [], 'file_references': [], 'extra_data': {}, 'dependencies': [{'purl': 'pkg:pypi/h5py', 'extracted_requirement': 'h5py', 'scope': 'setup', 'is_runtime': True, 'is_optional': False, 'is_resolved': False, 'resolved_package': {}, 'extra_data': {}}, {'purl': 'pkg:pypi/numpy', 'extracted_requirement': 'numpy', 'scope': 'setup', 'is_runtime': True, 'is_optional': False, 'is_resolved': False, 'resolved_package': {}, 'extra_data': {}}, {'purl': 'pkg:pypi/scipy', 'extracted_requirement': 'scipy', 'scope': 'setup', 'is_runtime': True, 'is_optional': False, 'is_resolved': False, 'resolved_package': {}, 'extra_data': {}}, {'purl': 'pkg:pypi/cython', 'extracted_requirement': 'cython', 'scope': 'setup', 'is_runtime': True, 'is_optional': False, 'is_resolved': False, 'resolved_package': {}, 'extra_data': {}}], 'repository_homepage_url': 'https://pypi.org/project/PyHEADTAIL', 'repository_download_url': 'https://pypi.org/packages/source/P/PyHEADTAIL/PyHEADTAIL-1.16.0.tar.gz', 'api_data_url': 'https://pypi.org/pypi/PyHEADTAIL/1.16.0/json', 'datasource_id': 'pypi_setup_py', 'purl': 'pkg:pypi/pyheadtail@1.16.0'}]}])
```

**4. Scanning that seame setup.py with SCTK works without a glitch with these deps**
```
dependencies:
- purl: pkg:pypi/h5py
extracted_requirement: h5py
scope: setup
is_runtime: yes
is_optional: no
is_resolved: no
resolved_package: {}
extra_data: {}
- purl: pkg:pypi/numpy
extracted_requirement: numpy
scope: setup
is_runtime: yes
is_optional: no
is_resolved: no
resolved_package: {}
extra_data: {}
- purl: pkg:pypi/scipy
extracted_requirement: scipy
scope: setup
is_runtime: yes
is_optional: no
is_resolved: no
resolved_package: {}
extra_data: {}
- purl: pkg:pypi/cython
extracted_requirement: cython
scope: setup
is_runtime: yes
is_optional: no
is_resolved: no
resolved_package: {}
extra_data: {}

```

**5. running this does not have the extra requires:**
`python-inspector --spec "xtrack==0.23.3" --json - -p 3.10 -o linux`

```
"resolved_dependencies_graph": [
{
"package": "pkg:pypi/cffi@1.15.1",
"dependencies": [
"pkg:pypi/pycparser@2.21"
]
},
{
"package": "pkg:pypi/lark@1.1.4",
"dependencies": []
},
{
"package": "pkg:pypi/numpy@1.23.5",
"dependencies": []
},
{
"package": "pkg:pypi/pycparser@2.21",
"dependencies": []
},
{
"package": "pkg:pypi/scipy@1.9.3",
"dependencies": [
"pkg:pypi/numpy@1.23.5"
]
},
{
"package": "pkg:pypi/xdeps@0.0.10",
"dependencies": [
"pkg:pypi/lark@1.1.4"
]
},
{
"package": "pkg:pypi/xobjects@0.1.27",
"dependencies": [
"pkg:pypi/cffi@1.15.1",
"pkg:pypi/numpy@1.23.5"
]
},
{
"package": "pkg:pypi/xpart@0.10.0",
"dependencies": [
"pkg:pypi/numpy@1.23.5",
"pkg:pypi/scipy@1.9.3",
"pkg:pypi/xobjects@0.1.27"
]
},
{
"package": "pkg:pypi/xtrack@0.23.3",
"dependencies": [
"pkg:pypi/numpy@1.23.5",
"pkg:pypi/scipy@1.9.3",
"pkg:pypi/xdeps@0.0.10",
"pkg:pypi/xobjects@0.1.27",
"pkg:pypi/xpart@0.10.0"
]
}
]
```

**6. we should have a way to return partial resolutions and we should not fail silently**

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.