HDFGroup / HDFGroup/h5pyd

Dask tests

Open
#51 4 comments 0 reactions 1 assignee Claimed by @jreadey View on GitHub
Dominant language
Python
Stars
124
Forks
46
Avg merge
1h 24m
Merged PRs (30d)
6

Description

As requested here by @mrocklin: https://github.com/pangeo-data/pangeo/issues/75#issuecomment-357734564

- [ ] Try XArray + Dask locally on the HSDS data to verify that it can be accessed concurrently from multiple threads
- [ ] Try XArray + Dask.distributed locally on the HSDS data to verify that the h5pyd objects can survive being serialized
- [ ] Try everything on a distributed cluster using KubeCluster and then look at the performance of scalable computing
- [ ] Try this all again on a cluster on S3, where presumably we would expect 100-200MB/s network access from each node.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.