remove_url_from_keys() strips characters needed by secret detection patterns
- Dominant language
- Python
- Stars
- 64
- Forks
- 38
- PR merge metrics
- No merged PRs in 30d
Description
While working on #63 (fixing corrupted regex patterns in keys_extractor), Copilot's review flagged a separate pre-existing issue: `remove_url_from_keys()` strips `.`, `_`, and `/` from the content *before* `keys_extractor()` runs, which means some valid detection patterns can't match.
Affected patterns:
- Google YouTube OAuth: needs `.apps.googleusercontent.com` (dots stripped)
- Google OAuth Access Token: needs `ya29.` prefix (dot stripped)
- Amazon MWS: needs `amzn.mws.` prefix (dots stripped)
- PayPal Braintree: needs `access_token` (underscore stripped)
- Slack Webhook: needs URL slashes (slashes stripped)
The special_chars list in `remove_url_from_keys()` currently includes `.`, `/`, and `_`, which are all characters that appear in real credential formats.
Possible fix: split the preprocessing into two steps — remove URLs and emails first (as it already does), then run `keys_extractor()` on that result before stripping the remaining special characters. The special character stripping is needed for `credential_extractor()` but shouldn't run before `keys_extractor()`.
Contributor guide
Research direction
Start at remove_url_from_keys() and follow its handoff to keys_extractor() and credential_extractor(). Check the listed Google, Amazon, PayPal, and Slack patterns, then verify that key detection retains required dots, underscores, and slashes while credential extraction still receives the intended stripped content.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- security
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100