drivendataorg / drivendataorg/cloudpathlib

Add integration test suite for cloudpathlib + other libraries

Open
#30 0 comments 0 reactions 0 assignees View on GitHub
tests
Dominant language
Python
Stars
628
Forks
88
Avg merge
17h 28m
Merged PRs (30d)
2

Description

There are probably some libraries that we want to test integration with, even it is not part of normal CI.

For example, this makes me remember that if pandas gets a path it will call `str` on it rather than using `.open`, etc. I think librosa also does the same.

In a previous iteration of similar code, I inspected the callstack to see if it was coming from pandas, and if so returned the local cache path on the call to `str` rather than the URL formulation. A little hacky, but not sure there is a better way if we want to only be able to pass a `CloudPath` rather than a buffer by doing `CloudPath.open()` (which, IIRC, does not work for all libraries since lots expect a path and don't handle buffers).

Another consideration is that some libraries may check `isinstance(x, Path)` and then call the correct methods like `.open`. We may want to subclass `Path` for this reason (the proper incantation of which I believe is `class CloudPath(type(PurePath())`).

- [ ] pandas (various file types)
- [ ] zarr
- [ ] xarray
- [ ] rasterio

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.