Erik-Debye / Erik-Debye/SWA-Scraper

Better solutions needed to escape bot detection

Open
#3 0 comments 0 reactions 0 assignees View on GitHub
enhancement help wanted
Dominant language
JavaScript
Stars
5
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Currently, the program works but definitely isn't perfect and does frequently get detected for being a bot. In particular, I have noticed that once the program gets detected, restarting the program at the point of detection does allow the program to continue.

My guess is that this is due to fingerprinting the chrome profile used, and when the program restarts, puppeteer creates a new temp profile and this escapes detection. Not sure where to start to find a fix.

Contributor guide

No contributing guide indexed for this repository

Research direction

No files or tests are named, and the issue does not define a specific fix. Start by tracing how Puppeteer creates and reuses the Chrome profile and how the program resumes after detection. Done requires a reproducible mitigation for bot detection with the scraper's existing workflow verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, supabase
Domain
backend, web-dev
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.