fsspec / fsspec/filesystem_spec
What to do with missing files in bulk operations
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 490
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 38
Description
Currently, if you do any of the bulk operations, if one or more of the targets encounters an error (e.g., file is missing), the overall process will fail. In the case of cp, this likely means the rest of the files got copied; in the case of mv, it may be that most files got copied, but none deleted (oh no!); in the case of cat, you would get no output, and in the case of get, you probably would get all the files that succeeded.
How to handle all of these cases? Should there be an "ignore errors" mode, or some other form of transaction, and what should the return from cat be (dict with only successful keys, dict with None or exception instance for failures, something else?)?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read the bulk cp, mv, cat, and get entry points and compare how each behaves when one target is missing. The issue leaves error handling and return-value semantics open, so completion requires an agreed design for partial failures before implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100