Unsuccessful recovery from ConnectionError
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 398
- Forks
- 181
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 36
Description
I'm trying to push a large-ish dataset to GCS via xarray/dask/zarr/gcsfs. Things are generally working during the setup and for the first part of the upload. However, after a bit, I'm getting a ConnectionError that is not recoverable. I'm pushing from a server at the University of Washington to bucket at "US-CENTRAL1". I would image the network at UW is pretty stable.
Version info:
xarray: jhamman:fix/zarr_set_attrs
dask: 0.16.0
zarr: master
gcsfs: master
Details of the full traceback are in this gist: https://gist.github.com/jhamman/25ddda993ad5b768e4b8289904be6779
cc @mrocklin, @martindurant
xref: https://github.com/pangeo-data/pangeo/issues/48, https://github.com/pydata/xarray/pull/1800
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Begin with the full traceback in the linked gist and the version information in the issue, then read the referenced pangeo issue and xarray pull request for related context. No repository file, test, or entry point is named; done means identifying why the large GCS upload becomes unrecoverable and establishing a verified recovery outcome.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, python
- Domain
- cloud, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100