Comcast / Comcast/xGitGuard

remove_url_from_keys() strips characters needed by secret detection patterns

Open
#64 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
64
Forks
38
PR merge metrics
No merged PRs in 30d

Description

While working on #63 (fixing corrupted regex patterns in keys_extractor), Copilot's review flagged a separate pre-existing issue: `remove_url_from_keys()` strips `.`, `_`, and `/` from the content *before* `keys_extractor()` runs, which means some valid detection patterns can't match.

Affected patterns:
- Google YouTube OAuth: needs `.apps.googleusercontent.com` (dots stripped)
- Google OAuth Access Token: needs `ya29.` prefix (dot stripped)
- Amazon MWS: needs `amzn.mws.` prefix (dots stripped)
- PayPal Braintree: needs `access_token` (underscore stripped)
- Slack Webhook: needs URL slashes (slashes stripped)

The special_chars list in `remove_url_from_keys()` currently includes `.`, `/`, and `_`, which are all characters that appear in real credential formats.

Possible fix: split the preprocessing into two steps — remove URLs and emails first (as it already does), then run `keys_extractor()` on that result before stripping the remaining special characters. The special character stripping is needed for `credential_extractor()` but shouldn't run before `keys_extractor()`.

Contributor guide

Open the contributing guide

Research direction

Start at remove_url_from_keys() and follow its handoff to keys_extractor() and credential_extractor(). Check the listed Google, Amazon, PayPal, and Slack patterns, then verify that key detection retains required dots, underscores, and slashes while credential extraction still receives the intended stripped content.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
security
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.