higlass / higlass/higlass-docker
Choose s3-fs-thing and implement
- Dominant language
- Shell
- Stars
- 34
- Forks
- 16
- PR merge metrics
- No merged PRs in 30d
Description
We'll need to store more data than fits comfortably on EBS. Options:
- EFS: but it's [expensive](https://aws.amazon.com/efs/pricing/): $0.30/GB-month. In their defense, it looks like they mirror between availability zones, and for some users that might be a big selling point.
- S3: [cheaper](https://aws.amazon.com/s3/pricing/): $0.023/GB-month
- https://github.com/s3fs-fuse/s3fs-fuse: but if you need less from your FS, points to:
- https://github.com/kahing/goofys: Check benchmarks: time to first byte is an order of magnitude smaller, and I think that's the critical measure for us.
- [Google Cloud Storage](https://cloud.google.com/storage/pricing): $0.02/GB-month: A bit cheaper, but I think we're comfortable with vendor lock-in?
- [Azure](https://azure.microsoft.com/en-us/pricing/details/storage/blobs/) in the same neighborhood.
Both [AWS](https://aws.amazon.com/s3/faqs/#sia_anchor) and [Google](https://cloud.google.com/storage/docs/storage-classes#nearline) offer a secondary tier for objects accessed less than once a month, and the cost is about half. Probably not worth the hassle, but it's there.
**Implementation**
- in the container? [possible](https://github.com/kahing/goofys/issues/118#issuecomment-262743941)... why not in production?
- in s3? Simpler, but configuration needs to be redone each time.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.