AlexsLemonade / AlexsLemonade/refinebio
Download SRA data from NCBI's S3
- Dominant language
- Python
- Stars
- 135
- Forks
- 21
- PR merge metrics
- No merged PRs in 30d
Description
### Context
```
If you haven't seen it, there is an API for looking for locations of files from SRA. It includes the ability to get data from the cloud as signed URLs. I've done some small tests and it appears to work reasonably well. There is also fusera that mounts a cloud bucket as a fuse file system. That also seems to work.
https://t.co/rn7OVxVBEe
```
### Problem or idea
We should use these to download data from SRA, this will let us go way faster.
### Solution or next step
We should probably look into using the Fusera thing to see how performance is when we mount the cloud bucket as a fuse file system. It seems like this could be awesome if can cut out the downloader step but less awesome if it bogs salmon down too much.
Contributor guide
No contributing guide indexed for this repository
Research direction
No implementation files or tests are named. Start by evaluating the NCBI SRA location API and Fusera-mounted S3 access, then measure whether cloud-based access improves download performance without bogging down Salmon; done means a decided and validated approach for replacing or retaining the downloader step.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud, data, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100