aboutcode-org / aboutcode-org/scancode.io

Improve maven pre-population

Abierto
#900 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
215
Forks
203
Merge medio
4 d 8 h
PR fusionados (30 d)
6

Descripción

As a follow up to https://github.com/nexB/scancode.io/issues/720 we need to handle the cases where there is no metadata available for a given JAR and therefore pre-propulation does not work at all like for a JAR without an embedded POM.

In these cases we could SHA1-match a JAR to a whole Maven index.

Using a whole maven index:

1. using minecode, index and keep up to date the purldb with the latest whole maven index (could be extended to other repos beyond Maven Central)
The outcome is that we can have "skinny" packages with only purl+sha1 for EVERYTHING in Maven central

2. In the populate pipeline:
2.1 collect packages and deps, and send these to purldb to index
2.2 also match JARs and poms to the purldb using the data in 1. and send a request to the purldb using a special endpoint to further index the skinny purl.

As an alternative to 1. we could also do focused crawled of other (smaller) Maven repos and also focus the crawling on a subset of a whole Maven repo, such as org/apache/tomcat

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.