aboutcode-org / aboutcode-org/scancode-toolkit

Improve false positive license detection for license lists

オープン
#2,651 コメント 12 件 リアクション 0 件 担当者 0 名 GitHub で見る
improve-license-detection license scan new feature
主要言語
Python
スター
2.6k
フォーク
791
平均マージ
1日 12時間
マージ済み PR(30日)
5

説明

We have lists of license identifiers in code or data files that are being detected and lead to many false positive (FP). These are typically list of SPDX identifiers and are mostly found in license-related tools... or package management tools. But these tools are seen everywhere.

See this attached example code from NuGet: [NuGetLicenseData.cs.txt](https://github.com/nexB/scancode-toolkit/files/7019923/NuGetLicenseData.cs.txt) with long lists like this:
```C#
{ "AGPL-1.0", new LicenseData(licenseID: "AGPL-1.0", isOsiApproved: false, isDeprecatedLicenseId: true, isFsfLibre: true) },
{ "AGPL-1.0-only", new LicenseData(licenseID: "AGPL-1.0-only", isOsiApproved: false, isDeprecatedLicenseId: false, isFsfLibre: false) },
{ "AGPL-1.0-or-later", new LicenseData(licenseID: "AGPL-1.0-or-later", isOsiApproved: false, isDeprecatedLicenseId: false, isFsfLibre: false) },
```

For now, adding new "false positive" license detection rules has been the solution to deal with list of license keys such as
- #1032
- #797
- #2647
- #2515

The problem is a bit related to
- #2403
... in the sense that the context of where we find a licenses matters: for bare words, the case, being surrounded by gibberish or found in a binary may be a false positive clue.

For instance the fact that we are in code literals like in #2502 could be a clue that this an FP

We should find a better way than just adding many new false positive rules like in 5f39252c3c817cb26bfbf7f93ceea08d8ab0c3a2 and #2505 ... some ideas:
- Heuristics when several licenses are detected in alphabetical order using most license names
- ML ?
- list of known packages Purl that have these issues (think about what would happen if you scan ScanCode :roll_eyes: )

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。