developmentseed / developmentseed/sat-ml-training
Getting big files into Colab without Download
- Dominant language
- Jupyter Notebook
- Stars
- 27
- Forks
- 14
- PR merge metrics
- No merged PRs in 30d
Description
Was working with the Zindi data, it's not really feasible for participants to download the data themselves and then upload to their google drives, could easily take hours. As a workaround we are putting all the data unzipped on Google Shared Drive accessible to participants. This is a common problem in general for ML work.
However it would be really awesome to use Colab to download the data directly to the cloud and unzip.
* It takes authentication to the Zindi platform to download, so using requests library is a little more complicated
* The links in the page are obsfucated, "Note: If you want to download the satellite data using a script, you can contact us for a permanent URL"
Alternatives:
* Get the Scenes from GEE or AWS Public Bucket
* Have the data ready to use (plan to do so anyways)
* There's some interesting Chrome plugins to save to drive directly, not sure which are safe to use.
This is a wishlist item if we've got all the other important stuff done already, but important to talk about.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.