drivendataorg / drivendataorg/cloudpathlib

Create API specifications for Backends/Clients

Open
#56 0 comments 0 reactions 0 assignees View on GitHub
documentation
Dominant language
Python
Stars
628
Forks
88
Avg merge
17h 28m
Merged PRs (30d)
2

Description

We don't have a full definition of what we rely on each backend function to do. For example, this just came up in #52 when we were discussing if `_download_file` should create non-existent parent directories.

Some example q's to specify:
- Should functions create non-existent parent directories or not? For most blob store backends, `_upload_file` won't complain if a portion of the "path" does not exist. However, for `_download_file`, it's usually the case that `pathlib` won't create non-existent parent directories for you.
- Should we overwrite existing files on the cloud or locally? (`_move_file`, `_upload_file`, `_download_file`)
- Should `_list_dir` include itself?
- Which functions should have "cloud native" implementations? (E.g., `_move_file` should prefer to issue commands to the backend rather than streaming files through the local machine).
- Should `_get_metadata` have a specified return format or should we just let the relevant `Path` class handle picking out what a user should want from a `dict`?
- Should we formalize `_is_file_or_dir` as a required method with enumerated responses?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.