aboutcode-org / aboutcode-org/scancode.io

Multi-level matching and lookup for package data

Open
#995 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
215
Forks
203
Avg merge
4d 8h
Merged PRs (30d)
6

Description

Assuming that there are multiple sources of curated, corrected or reviewed, I would like to have a pipeline that works with the PurlDB and these other sources of curated data.

- A first pipeline step would scan with ScanCode TK or match for packages with PurlDB.
- Then a second step would look a source of curated package data for each PURL and replace the package data with these data if the PURL is found.

It may be possible to invert the steps: for instance if this package data source supports some form of matching. For instance, when using ABOUT files as a source we can match based on paths or checksums. In this case it may be useful to perform this step first before matching to the PurlDB.

With this proposed approach, it will be possible to keep the PurlDB as a reference and the data source for matching and have a way to override, complete, correct or provide preferences (such as a license choice) from a curated data source.

As an example, the packages at https://repo1.maven.org/maven2/antlr/antlr/2.7.5/ have no license. An ABOUT or another datasource may have a proper license such as antlr-pd found from research. And this would allow to access this data and fill in the gaps.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.