dogsheep / dogsheep/github-to-sqlite

Copying very large repo (30k issues, 33k PRs)

Open
#84 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
474
Forks
50
PR merge metrics
No merged PRs in 30d

Description

I want to keep a local copy of metadata for qgis/QGIS, which is quite large with 30k issues and 33k pull requests. Has anyone used github-to-sqlite for anything like that?

The GitHub API has a rate limit of 5,000 API calls per hours, for authenticated users. And I think it takes one call for each issue - am I right? And github-to-sqlite does not offer only fetching only some subset of the issues, and furthermore does not commit to sqlite before all issues are fetched (so in case a call is interrupted, e.g. by the rate limit, the fetched data is simply discarded).

So I don't see any way that I can actually use github-to-sqlite for this (unless I pay for such privileges to github). Can you confirm?

Sincerely, Thomas

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.