Test / use semgrep for security CI
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 159
- Forks
- 156
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 16
Description
Currently this uses regex which can have a fair bit of false positives for some patterns, unless they are created with extremely complicated patterns.
Performing "semantic" pattern matching is basically the point of semgrep, so it can do thing like relate functions to modules, match function calls except when they're using literal strings (even when the literal string is first set on a local IIRC), etc... so it provides much more precise matching e.g. "execute() except when called on a literal string or an sql.SQL" is relatively easy to express.
The drawback is that semgrep is a bit expensive to run: running the two files I'm attaching takes 52 seconds wallclock, of which 172.40s user and 53.92s system.
The two files I'm attaching are the conversion of the existing patterns to semgrep rules, except for two which maybe should probably remain as regex. The files are encrypted with age/rage using your github ssh keys (I don't think the patterns are super secret as they're pretty easy to infer from the CI / runbot reports but...) @Xavier-Do @d-fence @odony @mart-e
It's called "tar.gz" because github still has not whitelisted age files, so I have to lie about the extension for the file to upload.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the current regex-based security CI and reviewing the attached rules.tar.gz conversion to semgrep rules. Compare the proposed semantic matching with the existing patterns, including the cases that may remain regex, and assess the reported runtime. Done should include a tested decision on whether semgrep can be used without unacceptable CI cost.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, security
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100