RefGenie - add assets currently used from AWS-iGenomes
@ErikDanielsson is already working on this.
Since Jun 17, 2022.
- Dominant language
- Python
- Stars
- 322
- Forks
- 255
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 5
Description
Split from #592, related directly to #1084
We want to transition away from using AWS-iGenomes to using the central RefGenie server(s) to host reference genome assets for the nf-core pipelines. To do this, we need to make sure that everything we currently have is available on a AWS refgenie server somewhere. Once all assets are mirrored we can do a clean swap in the config.
Going from igenomes.config, here is a check-list:
Summary of genomes
- GRCh37
- GRCh38
- GRCm38
- TAIR10
- EB2
- UMD3.1
- WBcel235
- CanFam3.1
- GRCz10
- BDGP6
- EquCab2
- EB1
- Galgal4
- Gm01
- Mmul_1
- IRGSP-1.0
- CHIMP2.1.4
- Rnor_6.0
- R64-1-1
- EF2
- Sbi1
- Sscrofa10.2
- AGPv3
- hg38
- hg19
- mm10
- bosTau8
- ce10
- canFam3
- danRer10
- dm6
- equCab2
- galGal4
- panTro4
- rn6
- sacCer3
- susScr3
Detailed version with asset types
- GRCh37
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- mito_name
- macs_gsize
- blacklist
- GRCh38
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- mito_name
- macs_gsize
- blacklist
- GRCm38
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- mito_name
- macs_gsize
- blacklist
- TAIR10
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- mito_name
- EB2
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- UMD3.1
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- mito_name
- WBcel235
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- mito_name
- macs_gsize
- CanFam3.1
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- mito_name
- GRCz10
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- mito_name
- BDGP6
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- mito_name
- macs_gsize
- EquCab2
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- mito_name
- EB1
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- Galgal4
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- mito_name
- Gm01
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- Mmul_1
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- mito_name
- IRGSP-1.0
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- mito_name
- CHIMP2.1.4
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- mito_name
- Rnor_6.0
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- mito_name
- R64-1-1
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- mito_name
- macs_gsize
- EF2
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- mito_name
- macs_gsize
- Sbi1
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- Sscrofa10.2
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- mito_name
- AGPv3
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- mito_name
- hg38
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- mito_name
- macs_gsize
- blacklist
- hg19
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- mito_name
- macs_gsize
- blacklist
- mm10
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- mito_name
- macs_gsize
- blacklist
- bosTau8
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- mito_name
- ce10
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- mito_name
- macs_gsize
- canFam3
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- mito_name
- danRer10
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- mito_name
- macs_gsize
- dm6
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- mito_name
- macs_gsize
- equCab2
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- mito_name
- galGal4
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- mito_name
- panTro4
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- mito_name
- rn6
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- mito_name
- sacCer3
- fasta
- bwa
- bowtie2
- star
- bismark
- readme
- mito_name
- macs_gsize
- susScr3
- fasta
- bwa
- bowtie2
- star
- bismark
- gtf
- bed12
- readme
- mito_name
From @nsheff:
For updating: We drive the server instances off a git repository, here: https://github.com/refgenie/refgenomes.databio.org
RIght now, it's semi-automated. Eventually we want it to be that you just update the repo with a PR, and when merged it will deploy automatically. For now, though, we have to build the things manually, but it's all scripted from that repository. The asses are all just annotated with a PEP. So, to add a genome you'd add it to this CSV file: https://github.com/refgenie/refgenomes.databio.org/blob/master/asset_pep/genome_descriptions.csv
Then you'd add whatever inputs are required to build the assets to this file: https://github.com/refgenie/refgenomes.databio.org/blob/master/asset_pep/recipe_inputs.csv
See also:
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.