AlexsLemonade / AlexsLemonade/refinebio

Switch index-refinery from storing .tar.gz files to .tar.xz files

Open
#79 0 comments 0 reactions 0 assignees View on GitHub
backlog optimization RNA-seq
Dominant language
Python
Stars
135
Forks
21
PR merge metrics
No merged PRs in 30d

Description

The `Homo_sapiens_short.tar.gz` file is 2.5 GB, whereas the same file compressed using `xz` is less than one GB. As these files don't actually feed into `salmon` in the gzipped format there's not much reason to use `gz` instead of `xz` other than the fact that `gz` may be easier for other users of the Index Refinery to use. However @jaclyn-taroni doesn't think that it would be a significant factor so `xz` seems superior.

Contributor guide

No contributing guide indexed for this repository

Research direction

No files, tests, or entry points are named. Locate the Index Refinery code that creates and stores the Homo_sapiens_short archive, then check how downstream consumers open it; the work is done when the archive is stored as .tar.xz without breaking supported consumers.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.