MeltanoLabs / MeltanoLabs/tap-github

Should the tap tests mocks github's API responses?

Open
#14 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
20
Forks
60
Avg merge
20h 29m
Merged PRs (30d)
6

Description

While developing I've run a few times into github's api rate limits (with my auth_token set!), which gets pretty annoying, as the tap does not work at all then.
This might become an issue for CI as well when the number of streams increases (the limit without an auth token is 60 requests/hour per IP).
So I am wondering if we should setup some sort of mocking of API calls, for instance using pytest's monkeypatching fixture, or some other mechanism.

Pros:

  • faster and more reliable test suite (so probably more tests written)
  • better development experience (running tests would be almost instant)

Cons:

  • it takes a little time to build the mock responses
  • if the actual API changes its format, the mocks need to be updated. Tools like VCR.py could be helpful with this.
  • breaking changes in the actual API would not break the tap's tests, but something downstream would probably fail. Unless we run a periodic CI job to compare mocks to actual API output.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review the existing tap tests and the GitHub API calls they exercise; the issue does not identify specific files or test entry points. Compare pytest monkeypatching with VCR.py, then define which API responses should be isolated and how the suite will verify that tests no longer depend on live rate-limited requests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, testing
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.