MeltanoLabs / MeltanoLabs/tap-gitlab
Tap downloads all global gitlab users
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 12
- Forks
- 29
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 1
Description
Hi there!
when attempting to get a single project, tap-gitlab attempts to download ALL users of gitlab.
In my case, I'm forwarding this to the target-gsheet and fill a google sheet with the project info. This is a free gitlab account, no pro license, no custom URL.
I am running the following command
tap-gitlab -c config_gitlab.json | target-gsheet -c config_gsheet.json
My config_gitlab.json looks like this
{
"api_url": "https://gitlab.com",
"private_token": "abcdefgandsoon",
"groups": "mygroup",
"projects": "mygroup/myproject",
"start_date": "2018-01-01T00:00:00Z",
"ultimate_license": false,
"fetch_merge_request_commits": false,
"fetch_pipelines_extended": false,
"fetch_group_variables": false,
"fetch_project_variables": false
}
Upon running the command I get this output
INFO Starting sync
INFO Skipping stream: merge_request_commits
INFO Skipping stream: epics
INFO Skipping stream: epic_issues
INFO Skipping stream: pipelines_extended
INFO GET https://gitlab.com/api/v4/users
INFO GET https://gitlab.com/api/v4/users
...after which a 'site_users' panel appears in the google sheet which is filled with endless rows of users.
Am I doing something wrong so that tap-gitlab doesn't filter users by group?
Or is there an option to skip users alltogether (I don't actually need them)
Best
Benjamin
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the command with the supplied config and observe the site_users stream and repeated GET https://gitlab.com/api/v4/users requests. Trace where that stream is enabled or scoped, then verify that a single project does not download all GitLab users or that users can be skipped when they are not needed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100