dchaley / dchaley/qupath-project-initializer

Determine disk sizing needs

Open
#40 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Kotlin
Stars
1
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Today, the qupath code uses the disk for the following:
- download ALL input images
- download ALL segmentation masks (nucleus + whole cell)
- qupath project files: especially the data.qpdata files (which store cells/ROIs + associated measurement data)

For large enough images, this quickly exceeds the 30GB default boot disk.

As we did in https://github.com/dchaley/deepcell-imaging/issues/305, we could figure out the space requirements then add the appropriate disk.

However we also believe the above approach might change depending on how we scale the process. (It takes 12.5 hours to measure a ~140M pixel image, which is way too slow!) So we aren't sure what the sizing will be.

A temporary solution could be to dramatically over-provision storage for now. Balanced PDs in us-central1 cost $0.10 per GB per month. So, 500 GB would cost $50 for the month. If we "only" ran for 0.5 days, that would be ~$0.84. That's cheap enough to not worry about it while we scale up.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.