dependency-check / dependency-check/DependencyCheck

feat: enhance lucene index stop words when found in lowercase tokens

Open
#5,443 0 comments 0 reactions 1 assignee Claimed by @jeremylong View on GitHub
enhancement
Dominant language
Java
Stars
7.7k
Forks
1.4k
Avg merge
9d 22h
Merged PRs (30d)
13

Description

Currently, if a CPE contains `framework`, or other things we consider a stop word, the stop word may not be identified if it is concatenated with another word and it is all lowercase. For instance, 'yiiframework` when processed would not contain a token `yii` because there is nothing that identifies the stop word within the broader token.

A new `LowerCaseStopWordTokenizingFilter` should be added to identify when a larger stop word (package, framework, etc) is contained in a token and remove the stop word.

This has the potential to create more false positives that we would need to work through and update the suppression files to account for new common FP.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.