singer-io / singer-io/tap-gitlab
429 Too Many Requests
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 16
- Forks
- 18
- PR merge metrics
- No merged PRs in 30d
Description
I used stitchdata.com to extract my Gitlab data and stored to other DB.
There are > 100 projects defined in my gitlab.
But I always end up with "429 Too Many Requests" error when running the sync process.
This is my log :
2019-01-30 17:02:45,456Z tap - INFO GET https://gitlab.com/api/v4/projects/XXXX/repository/commits?private_token=....................&page=27
2019-01-30 17:02:45,526Z tap - INFO GET https://gitlab.com/api/v4/projects/XXXX/repository/commits?private_token=....................&page=28
2019-01-30 17:02:45,600Z tap - INFO GET https://gitlab.com/api/v4/projects/XXXX/repository/commits?private_token=....................&page=29
2019-01-30 17:02:45,699Z tap - INFO GET https://gitlab.com/api/v4/projects/XXXX/repository/commits?private_token=....................&page=30
2019-01-30 17:02:47,715Z tap - CRITICAL Error making request to GitLab API: GET https://gitlab.com/api/v4/projects/XXXX/repository/commits?private_token=....................&page=30 [429 - b'<!DOCTYPE html>\n<html>\n<head>\n <meta content="width=device-width, initial-scale=1, maximum-scale=1" name="viewport">\n <title>429 Too Many Requests</title>
Any one can help?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the sync against a GitLab account with more than 100 projects and inspect requests to the GitLab commits endpoint, especially around page 30. The issue does not name a source file, test, or expected handling for HTTP 429 responses, so the completed behavior is not defined.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100