aboutcode-org / aboutcode-org/scancode.io

Improve maven pre-population

Đang mở
#900 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Python
Star
215
Fork
203
Merge trung bình
4 ngày 8 giờ
Pull request đã merge (30 ngày)
6

Mô tả

As a follow up to https://github.com/nexB/scancode.io/issues/720 we need to handle the cases where there is no metadata available for a given JAR and therefore pre-propulation does not work at all like for a JAR without an embedded POM.

In these cases we could SHA1-match a JAR to a whole Maven index.

Using a whole maven index:

1. using minecode, index and keep up to date the purldb with the latest whole maven index (could be extended to other repos beyond Maven Central)
The outcome is that we can have "skinny" packages with only purl+sha1 for EVERYTHING in Maven central

2. In the populate pipeline:
2.1 collect packages and deps, and send these to purldb to index
2.2 also match JARs and poms to the purldb using the data in 1. and send a request to the purldb using a special endpoint to further index the skinny purl.

As an alternative to 1. we could also do focused crawled of other (smaller) Maven repos and also focus the crawling on a subset of a whole Maven repo, such as org/apache/tomcat

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.