MeltanoLabs / MeltanoLabs/tap-github

Experiencing 401 Bad Credentials when credentials are valid

Open
#243 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
20
Forks
60
Avg merge
20h 29m
Merged PRs (30d)
6

Description

While synchronising data for our github organisation, we've been experiencing consistent HTTP 401 errors with the reason "Bad credentials" which should occur when an invalid token is provided.

Although, this happens after a lot of time into the synchronisation, after performing multiple successful API calls with the same credentials. According to [this line of code](https://github.com/dlouseiro/tap-github/blob/ecc495cb56d76f86053966cd2e92f0d8ec9bbe16/tap_github/client.py#L228) a similar behaviour has been experienced (random 401 errors - triggering a retry), but not with the reason "Bad Credentials".

All failures experienced were while fetching team members stream, but always in different teams/members. Not consistently on the exact same call.

While trying a curl call to [this endpoint](https://docs.github.com/en/rest/teams/members?apiVersion=2022-11-28#get-team-membership-for-a-user) using the same token and endpoint call that cause the failure, we receive a 200 OK.

Yet another detail which makes the behaviour even weirder is that if I do an execution with `--select teammembers.*`, the execution is successful without any failure. With the exact same token.

While an easy solution could be to change[this line of code](https://github.com/dlouseiro/tap-github/blob/ecc495cb56d76f86053966cd2e92f0d8ec9bbe16/tap_github/client.py#L228) to retry on any 401 call (not excluding bad credentials), this would cause the tap to retry, even when a bad token is effectively provided.

So, the only solution I see would be to retry on every 401 call, as long as an X amount of API calls was already successfully performed with the same credentials.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in tap_github/client.py around line 228, where 401 responses are retried, and trace how the team members stream makes its API calls. Compare full synchronisation with --select teammembers.* and determine how retry behavior should distinguish invalid credentials from intermittent failures after successful calls; done means the reported scenario is handled without masking genuinely bad tokens.

Written by the indexing model from the issue text.

Assessment

Tech stack
github, python
Domain
api, data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.