aboutcode-org / aboutcode-org/purldb

Report missing corresponding source code for a package

Aberta
#185 0 comentários 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
HTML
Estrelas
67
Forks
69
Merge médio
8d 8h
PRs com merge (30d)
1

Descrição

Reported from a private chat:

> Having an option to note if there was a package found that does not have known source code would be very useful.

> Even more so, if it also included a simple way to send a suggestion to PurlDB if the user of SCIO happens to know the origin of the source code.

For this here are a few things to consider:

1. In SCIO with the d2d pipeline, we are matching deployed binaries to source code: if this does not happen we surface this there. We do not do this d2d pipeline for packages in the PurlDB (and this would need to be a new streamlined pipeline dedicated to each pipeline type for this)

2. In the PurlDB: For third-party packages, we have created specific code that looks hard for the corresponding source code of a JAR and then uses the scans from these to expand license and copyright from source to binaries using the new "package set" feature.

3. We should in the future also perform the step 1. on all the packages to ensure that we have the corresponding sources of all the binary packages.

The 3. point is important as I see more and more:
- public JARs that are in fact Uberjars combining other JARs
- npm that are in fact webpacked bundles combining other npms

Step 1. is done in ScanCode.io in the develop_to_deploy pipeline

Step 2. is done in the PurlDB

Note that finding the source code is surprisingly involved: see for instance https://github.com/nexB/purldb/blob/main/packagedb/find_source_repo.py to find the corresponding Git repo and commit of a package.

So in short, the current status is:
- we map all the binaries to sources and flag these that are missing in SCIO d2d. BUT we do not do this for PurlDB packages. Yet.
- we try to collect the corresponding sources of the third-party binaries

- Q: What happens if source code match is not in PurlDB? (edited)
- A: For now, we are trying hard to get it at indexing time, but nothing happens if we do not have it. We could query and report a package set that does not contain a source code repo and/or source archive package https://github.com/nexB/purldb/blob/main/packagedb/models.py#L443 are the types we have there

Possible implementations could be to:
- add a flag to the API results if we are missing sources in a package set
- OR list all the set packages and their type when doing a package API call

And this would need to be stored and surfaced in SCIO scans so we know when source code may be missing.

As for the addition of a known source, this could be an API endpoint that adds a new package and states:
- its purl (and download URL and other URLs as needed)
- its [content type](https://github.com/nexB/purldb/blob/8428cb8ab00995f1bc0e4d2b48ecd67fa1126a80/packagedb/models.py#L443)
- which package set it belongs

Once available it could be exposed with some simple UI in SCIO in the context of the packages list or of a package detail.

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.