scrapy/protego
View on GitHubA pure-Python robots.txt parser with support for modern conventions.
- Stars
- 94
- Forks
- 34
- Open beginner issues
- 0
- Indexed issues
- 1
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 8
- Dominant language
- DIGITAL Command Language
- License
- BSD-3-Clause
- Last GitHub push
- Sep 19, 2026
- Latest indexed
- Sep 19, 2026
- Contributing guide
- No contributing guide
- Code of conduct
- No code of conduct
- Beginner labels
- No beginner labels indexed
1 open issue indexed
Loading issues