aboutcode-org / aboutcode-org/scancode-toolkit

License stopwords problems

Open
#4,238 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
2.6k
Forks
791
Avg merge
1d 12h
Merged PRs (30d)
5

Description

The issue first showed up when generating new "is_required_phrase" rules or updating rules to become one.

Some short rules are for license keys or names and they can contain stopwords. For instance:
- in the "h2-1.0" license where "h2" is a stopword.
- in liliq-p-1.1 and cern-ohl-p-2.0, "p" is a stopword
- in "Server Side Public License", "side" is a stopword
- in "LGPL licensed bash script", "script" is a stopword

There are a few ways to consider to resolve this:
1. remove some longer stop words like side and script
2. abandon using stop words entirely
3. just ensure that some required phrase rules are not generated/updated as such and accept a tiny bit of inaccuracy for these the few seldom seen rules
(note that these license ids will always be matched in license expressions since that does not ignore stopwords)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.