dandi / dandi/dandi-cli

upload: if any file/path is skipped due to not part of DANDI layout -- log WARNING upon completion or ask

Open
#1,493 0 comments 1 reaction 0 assignees View on GitHub
cmd-upload UX
Dominant language
Python
Stars
28
Forks
37
Avg merge
1d 17h
Merged PRs (30d)
9

Description

Might need to be implemented along with

- https://github.com/dandi/dandi-cli/issues/119

and overall provide partial (UX) resolution to e.g.

- https://github.com/dandi/dandi-cli/issues/1464

where "non-DANDI layout" file is found to be not uploaded.

I think overall implementation, upon processing command upload for a specific (sub)`path` (could be top of dandiset), and having obtained list of paths which were uploaded, should then call `find_unused_paths(path, used_paths)`, which would `os.walk` that `path`, and if encountering folder, check if any path among `user_paths` is under that folder path, if not -- add entire folder path into result, if some is, descend into that folder (recurse); if encountering a non-folder, see if was among `used_paths` and if not -- add to result.

Common known to be ignored paths (like vcs folders and some OSX trash folders) should be ignored altogether.

Then WARNING could report total number of skipped entities (folders of files) with some (up to 10?), and log the rest at DEBUG level (into log file).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.