JustAnotherArchivist / JustAnotherArchivist/snscrape

Thread safety

Open
#623 4 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
5.4k
Forks
783
PR merge metrics
No merged PRs in 30d

Description

Since people seem to keep trying to use snscrape with threads (despite this not being listed as a feature anywhere) and running into problems (seemingly without searching the issues)...

**snscrape is currently not thread-safe.**

I'd like to evaluate at some point whether it's easy enough to make snscrape thread-safe. One known issue is the Twitter module's guest token manager. Testing thread safety will be an issue, too.

Relevant prior issues: #307 #584 #622

(SEO keywords: threading multithreading)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing prior issues #307, #584, and #622, then inspect the Twitter module's guest token manager, identified as one known thread-safety issue. Determine the scope of making snscrape thread-safe and how thread-safety can be tested; done means the required behavior and test coverage are agreed and implemented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.