coreruleset / coreruleset/coreruleset
Quantitative testing: Round 2 of testing and squashing natural language false positives
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.3k
- Forks
- 470
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 11
Description
This is a continuation of the quantitative testing project from the 2025 Dev Retreat.
Description
We worked to squash natural language FPs during and after the 2023 Dev Retreat (round 1 issue: #3392).
Unfortunately, as far as we can tell, when we introduced the Go-based quantitative testing as a GitHub action it stopped working (it would always say "everything is fine" even though we've had many PRs introducing new natural language false positives.)
We now need to revisit this and squash these natural language FPs again.
Current results
| Hits | Rule ID | Paranoia Level |
|---|---|---|
| 203 | 932236 | PL2 |
| 121 | 942200 | PL2 |
| 52 | 942131 | PL2 |
| 28 | 932250 | PL1 |
| 28 | 932230 | PL1 |
| 24 | 932235 | PL1 |
| 20 | 942440 | PL2 |
| 15 | 942390 | PL2 |
| 14 | 932200 | PL2 |
| 12 | 931130 | PL2 |
| 9 | 932240 | PL2 |
| 7 | 942410 | PL2 |
| 6 | 942150 | PL2 |
| 6 | 932231 | PL2 |
| 5 | 942380 | PL2 |
| 4 | 942430 | PL2 |
| 3 | 942520 | PL2 |
| 3 | 942210 | PL2 |
| 2 | 942400 | PL2 |
| 2 | 942370 | PL2 |
| 2 | 932271 | PL2 |
| 1 | 942521 | PL2 |
| 1 | 942360 | PL1 |
| 1 | 942230 | PL1 |
| 1 | 942180 | PL2 |
| 1 | 942100 | PL1 |
| 1 | 933160 | PL1 |
Open sub-issues
The following sub-issues are open in relation to this main task:
- #4356
- Natural language FPs with rule 932230 PL1 #XXXX
- Natural language FPs with rule 932235 PL1 #XXXX
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the quantitative testing GitHub Action, the 2023 round-one issue #3392, and sub-issue #4356. The issue does not name files or tests; completion would require restoring detection of natural-language false positives and addressing the listed rule hits and open sub-issues.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, go
- Domain
- ci-cd, security, testing-qa
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100