galaxyproject / galaxyproject/brc-analytics

Load all catalog input data via dlt

Open
#1,686 0 comments 0 reactions 1 assignee Claimed by @hunterckx View on GitHub
Dominant language
TypeScript
Stars
7
Forks
11
Avg merge
3d 15h
Merged PRs (30d)
15

Description

In preparation for porting transformations to be done in a database via dbt, we can load input data into the database via dlt (from where it can, for now, be read back out and transformed via Pandas).

Inputs to include:

- Full source lists from catalog YAML files (assemblies/organisms/outbreaks)
- NCBI genomes
- SRA metadata? (Not used by either catalog)
- UCSC assemblies
- Gene model URLs
- Datacache directories
- Organism images

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.