HashSlap-Summer-of-Code / HashSlap-Summer-of-Code/scraping-scripts
🌐 Build a Dynamic Web Scraper with Proxy & User-Agent Rotation
Open
advanced
hacktoberfest
hssoc
- Dominant language
- Python
- Stars
- 0
- Forks
- 4
- PR merge metrics
- No merged PRs in 30d
Description
Description:
Create a robust scraper that:
Accepts any URL as input
Rotates between multiple user-agents and proxy IPs
Detects & handles bot detection pages (like CAPTCHA or 403 errors)
🔧 Requirements:
Use libraries like requests, random, and optionally scrapy
Proxy list can be free or local
Log failed attempts and retry logic
📁 Suggested Folder: dynamic-rotating-scraper/
Contributor guide
Assessment
This issue has not been assessed yet.