aboutcode-org / aboutcode-org/scancode-toolkit

Improve PyPI package declared license detection

Abierto
#2,487 0 comentarios 0 reacciones 0 asignados Ver en GitHub
improve-license-detection license scan new feature package scan package-formats
Lenguaje dominante
Python
Estrellas
2.6k
Forks
791
Merge medio
1 d 12 h
PR fusionados (30 d)
5

Descripción

The goal of this ticket is to improve PyPI package license detection across the board. While scancode-toolkit's PyPI package detection is pretty good, there a few repeat cases where license information is not properly gathered from PyPI package metadata. Usually this is because a `declared_license` value contains things we did not expect (like a URL) or is improperly formed.

Resolving this would likely require a mix of:
- adding new license detection rules to scancode,
- adding new and improved code to handle the specific patterns of license,
- creating new license mappings
- and possibly working with upstream maintainers to improve their license declarations.

The approach should be to start with a complete data set of all package manifests and find patterns of license issues and establish the baseline, possibly with classifiers and ML. The end results should be a significant improvement to the license detection quality for the PyPI packages.

This https://github.com/pypa/bandersnatch/ and the PyPI API may help collect a list of all declared licenses.

See also https://www.python.org/dev/peps/pep-0639/ and may be https://github.com/nexB/scancode-toolkit/issues/253 too

There are also other related ticket for other package types such as:
- https://github.com/nexB/scancode-toolkit/issues/2412 for RPM that has some detailed examples

And a project idea: https://github.com/nexB/aboutcode/wiki/Project-Ideas-Improve-PyPI-package-license-detection

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.