JustAnotherArchivist / JustAnotherArchivist/snscrape

Likelihood of rate limits/IP ban from Twitter?

Open
#385 10 comments 1 reaction 0 assignees View on GitHub
module:twitter question
Dominant language
Python
Stars
5.4k
Forks
783
PR merge metrics
No merged PRs in 30d

Description

I'm currently trying to use snscrape to download Tweets from Twitter. According to my calculations, I should be getting around 2,200,000 Tweets in total by the time it finishes. I'm concerned about the possibility of getting IP banned from Twitter as a result of this. Is this something worth being concerned about, or should I not worry?

More generally:
- How much scraping should be safe to do before worrying about getting rate-limited or IP banned?
- If the concern of getting IP banned is valid, do these bans typically go away after a period of time, or are they permanent, etc?

This tool seems like a godsend, compared to the limits of the official Twitter API. Having a more solid understanding of the "safe zone" would make me feel more comfortable with using it. I know that maintainers can't *guarantee* anything about rate limits or IP bans, but if anyone has experience with where they begin to set in, knowing that would help a lot!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.