singer-io / singer-io/tap-github

Emitted schemas do not honor catalog

Open
#153 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
76
Forks
86
PR merge metrics
No merged PRs in 30d

Description

I'm running this tap via meltano. I'm hitting the same bug as https://github.com/singer-io/tap-github/issues/130, so I was thinking I could work around this by providing a custom catalog to the tap, via meltano.yml

    select:
      - issues.id
      - issues.url
      - issues.repository_url
      - issues.state
      - issues.title
      - issues.number
      - issues.comments
      - issues.closed_at

The singer record messages emitted by the tap only include the right fields. However, it appears the schema message does not honor the catalog and still includes the problematic issues.assignee field. This means I'm not able to connect tap-github to target-bigquery due to the missing issue.assignee schema (even though this property is not selected!)

I'm new to the singer ecosystem, so I have checked other taps and they seem to adapt the schema message they emit based on the catalog provided to the tap. But I may be getting this wrong, so I appreciate your feedback if this is a problem with this tap or if I need to look elsewhere.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue using the custom catalog and select entries shown in the issue, then inspect tap-github's schema-message emission path. Compare the emitted schema with the selected fields and verify that the unselected issues.assignee field is absent, including when consuming the output with target-bigquery.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.