MeltanoLabs / MeltanoLabs/Singer-Most-Wanted

tap-test-data

Open
#90 6 comments 1 reaction 1 assignee View on GitHub

@pnadolny13 is already working on this.

Since Sep 12, 2023.

Dominant language
No language data
Stars
5
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Many taps have statically defined schemas so you can --discover without credentials to get the catalog schema for all streams available. Given that catalog as input it would be nice to have a tap that generates fake data that conforms to it. This would be very helpful for testing pipelines without credentials. https://github.com/lk-geimfari/mimesis or https://github.com/joke2k/faker can be used to generate fake data.

This is somewhat similar to https://github.com/meltano/sdk/issues/1009 but that requires the tap to pull real data in order to mask it.

@edgarrmondragon any thoughts? I want to test out https://hub.meltano.com/extractors/tap-amazon-sp but I dont have credentials which made me wonder if we could generate data as if I do. This would unlock anything downstream and would expect that the tap conforms to its own schema, otherwise thats a bug anyways.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.