awslabs / awslabs/git-secrets

Add *hashed* file version to .gitallowed? So future changes get caught containing secrets again

Open
#246 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
13.4k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

Would you be open to adding / accepting a PR for some way to record git-secrets exceptions to a repo (a la `.gitallowed`), except rather than just matching by filename and line contents, it can be whitelisted by _hash_? (i.e. the output of `git hash-object`)

We find ourselves wanting to _remove_ some exceptions that have been made in the past to some IPython notebooks (which are ultimately JSON blobs with newlines). These contain lines that match some custom secrets regexes we've defined (9-digit numbers which could be SSNs 🙂), but we know they're not actually SSNs.

Up to this point we've just listed these files by name in `.gitallowed` per https://github.com/awslabs/git-secrets#ignoring-false-positives, but we've developed an interest in reducing the risk that somebody will _modify_ those files and accidentally introduce secrets later on (which will not be flagged as containing secret data because it's whitelisted by filename, so they might not think to double check). We'd prefer that somebody have to manually re-whitelist a file that's flagged as potentially containing a secret _any_ time they're committing changes to it.

Any thoughts?

---

I don't see a way to do this backwards compatibly within the `.gitallowed` file itself, so the first approach that comes to mind is checking for another file called `.gitallowed-hashes` or something like that... and if a tree or blob SHA appears on a line in that file then it gets ignored by git-secrets. We could then (maybe) add a command to `git-secrets` to _add_ a directory/blob to the list based on its current contents, with the path to the directory/blob listed after the hash in a comment string.

So e.g. what I envision:

```console
$ git secrets --add "[0-9]{9}"

$ echo 123456789 > my-scary-file

$ git secrets --scan
# comes up with error

$ echo "$(git hash-object my-scary-file) # my-scary-file" >> .gitallowed-hashes

$ git secrets --scan
# comes up clean

$ echo "new evil secret for real this time 089291203" >> my-scary-file

$ git secrets --scan
# comes up with error
```

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing .gitallowed exception behavior and the git-secrets --scan and --add entry points. Compare how a .gitallowed-hashes file could use git hash-object values for files or trees. Done means unchanged whitelisted content is ignored while any modified content is scanned and requires a new hash exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, shell
Domain
devtools, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.