drivendataorg / drivendataorg/cloudpathlib
Create API specifications for Backends/Clients
- Dominant language
- Python
- Stars
- 628
- Forks
- 88
- Avg merge
- 17h 28m
- Merged PRs (30d)
- 2
Description
We don't have a full definition of what we rely on each backend function to do. For example, this just came up in #52 when we were discussing if `_download_file` should create non-existent parent directories.
Some example q's to specify:
- Should functions create non-existent parent directories or not? For most blob store backends, `_upload_file` won't complain if a portion of the "path" does not exist. However, for `_download_file`, it's usually the case that `pathlib` won't create non-existent parent directories for you.
- Should we overwrite existing files on the cloud or locally? (`_move_file`, `_upload_file`, `_download_file`)
- Should `_list_dir` include itself?
- Which functions should have "cloud native" implementations? (E.g., `_move_file` should prefer to issue commands to the backend rather than streaming files through the local machine).
- Should `_get_metadata` have a specified return format or should we just let the relevant `Path` class handle picking out what a user should want from a `dict`?
- Should we formalize `_is_file_or_dir` as a required method with enumerated responses?
Contributor guide
Assessment
This issue has not been assessed yet.