dependency-check / dependency-check/DependencyCheck
feat: enhance lucene index stop words when found in lowercase tokens
- Dominant language
- Java
- Stars
- 7.7k
- Forks
- 1.4k
- Avg merge
- 9d 22h
- Merged PRs (30d)
- 13
Description
Currently, if a CPE contains `framework`, or other things we consider a stop word, the stop word may not be identified if it is concatenated with another word and it is all lowercase. For instance, 'yiiframework` when processed would not contain a token `yii` because there is nothing that identifies the stop word within the broader token.
A new `LowerCaseStopWordTokenizingFilter` should be added to identify when a larger stop word (package, framework, etc) is contained in a token and remove the stop word.
This has the potential to create more false positives that we would need to work through and update the suppression files to account for new common FP.
Contributor guide
Assessment
This issue has not been assessed yet.