clearlydefined / clearlydefined/crawler

Unknown LicenseRef issue with a lot of packages

Open
#636 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
60
Forks
38
Avg merge
1h 49m
Merged PRs (30d)
1

Description

Hi!

As can be seen in https://github.com/clearlydefined/curated-data/pull/29516 and on other popular libraries like [jetty](https://clearlydefined.io/definitions/maven/mavencentral/org.eclipse.jetty/jetty-util/12.0.18), there's an ongoing issue where a lot of popular packages are being detected with a `LicenseRef-scancode-unknown-license-reference`.

I thought it might have been a [scancode issue](https://github.com/aboutcode-org/scancode-toolkit/issues/4222) but it seems like the issue is on the usage of the tool by clearlydefined. Digging into recent commits, it seems like it might be caused by https://github.com/clearlydefined/service/issues/1149, and its related merged pull requests. This [comment thread on a PR](https://github.com/clearlydefined/service/pull/1056#discussion_r1551607561) actually raises the same issue that I'm reporting here.

This issue is causing a lot of problems in our automated license check tool since `LicenseRef-scancode-unknown-license-reference` is obviously not in our allow list.

Thanks for looking into this!

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reported LicenseRef-scancode-unknown-license-reference result with the jetty example and compare the crawler's ScanCode usage against service issue 1149 and the discussion in pull request 1056. Done means popular packages no longer receive the erroneous unknown-license reference and the existing automated license checks can accept the corrected results.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.