meta-pytorch / meta-pytorch/data
Caching on dir level does not guarantee cache consistency
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 179
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 2
Description
Seems to be invalid use-case for me. What if code will fail at the middle of extraction and target dir would have only half of the files. Next cache pass would think that cache is properly propagated and will fail to find some of the pieces.
I have no better options so far (as tar unpacking seems to be valid use-case), but lets keep tracking this.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with test/test_remote_io.py around line 155 and reproduce the directory-level cache behavior described here. Define what should happen when extraction stops halfway, then use that expected behavior to determine whether the test remains valid and what regression coverage is needed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- data
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100