aboutcode-org / aboutcode-org/scancode.io

Improve maven pre-population

Aperta
#900 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
215
Fork
203
Merge medio
4g 8h
PR unite (30g)
6

Descrizione

As a follow up to https://github.com/nexB/scancode.io/issues/720 we need to handle the cases where there is no metadata available for a given JAR and therefore pre-propulation does not work at all like for a JAR without an embedded POM.

In these cases we could SHA1-match a JAR to a whole Maven index.

Using a whole maven index:

1. using minecode, index and keep up to date the purldb with the latest whole maven index (could be extended to other repos beyond Maven Central)
The outcome is that we can have "skinny" packages with only purl+sha1 for EVERYTHING in Maven central

2. In the populate pipeline:
2.1 collect packages and deps, and send these to purldb to index
2.2 also match JARs and poms to the purldb using the data in 1. and send a request to the purldb using a special endpoint to further index the skinny purl.

As an alternative to 1. we could also do focused crawled of other (smaller) Maven repos and also focus the crawling on a subset of a whole Maven repo, such as org/apache/tomcat

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.