aboutcode-org / aboutcode-org/scancode-toolkit

Improve npm package declared license detection

Open
#2,488 0 comments 0 reactions 0 assignees View on GitHub
improve-license-detection license scan new feature package scan package-formats
Dominant language
Python
Stars
2.6k
Forks
791
Avg merge
1d 12h
Merged PRs (30d)
5

Description

The goal of this ticket is to improve npm package license detection across the board. While scancode-toolkit's npm package detection basics are OK, there a few repeat cases where license information is not properly gathered from npm package metadata. Usually this is because a `declared_license` value contains things we did not expect (like a URL) or is improperly formed. There are also specific things we do not support such as `SEE LICENSE` (defined in https://docs.npmjs.com/cli/v7/configuring-npm/package-json#license and related ticket in https://github.com/nexB/scancode-toolkit/issues/1364 )

Resolving this would likely require a mix of:
- adding new license detection rules to scancode,
- adding new and improved code to handle the specific patterns of license,
- creating new license mappings
- and possibly working with upstream maintainers to improve their license declarations.

The approach should be to start with a complete data set of all package manifests and find patterns of license issues and establish the baseline, possibly helped by heuristics, classifiers and ML if needed. The end results should be a significant improvement to the license detection quality for the npm packages.

See also these projects and pages of interest:
- https://github.com/jslicense/spdx-correct.js
- https://github.com/jslicense/npm-license-corrections.json
- https://github.com/nice-registry/
- https://github.com/nice-registry/npm-license-stats and https://github.com/nice-registry/npm-license-stats/blob/master/results.json in particular that may have all the npm licenses

There are also other related ticket for other package types such as:
- https://github.com/nexB/scancode-toolkit/issues/2412 for RPM that has some detailed examples
- https://github.com/nexB/scancode-toolkit/issues/2487 for PyPI

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.