drivendataorg / drivendataorg/cloudpathlib
Add integration test suite for cloudpathlib + other libraries
- Dominant language
- Python
- Stars
- 628
- Forks
- 88
- Avg merge
- 17h 28m
- Merged PRs (30d)
- 2
Description
There are probably some libraries that we want to test integration with, even it is not part of normal CI.
For example, this makes me remember that if pandas gets a path it will call `str` on it rather than using `.open`, etc. I think librosa also does the same.
In a previous iteration of similar code, I inspected the callstack to see if it was coming from pandas, and if so returned the local cache path on the call to `str` rather than the URL formulation. A little hacky, but not sure there is a better way if we want to only be able to pass a `CloudPath` rather than a buffer by doing `CloudPath.open()` (which, IIRC, does not work for all libraries since lots expect a path and don't handle buffers).
Another consideration is that some libraries may check `isinstance(x, Path)` and then call the correct methods like `.open`. We may want to subclass `Path` for this reason (the proper incantation of which I believe is `class CloudPath(type(PurePath())`).
- [ ] pandas (various file types)
- [ ] zarr
- [ ] xarray
- [ ] rasterio
Contributor guide
Assessment
This issue has not been assessed yet.