meta-pytorch / meta-pytorch/data

FileExistsError when using `on_disk_cache` and multiple workers

Open
#1,186 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.3k
Forks
179
Avg merge
6d 1h
Merged PRs (30d)
2

Description

🐛 Describe the bug

When multiple workers try to create a folder structure at the same time it fails and raises FileExistsError.

Expected behavior: create folder structure and continue

torchdata.datapipes.iter.IterableWrapper(urls)\
        .on_disk_cache(filepath_fn=filepath_fn)\
        .open_files_by_fsspec(mode='rb')\
        .end_caching(filepath_fn=filepath_fn)
        
dl = DataLoader(datapipe, batch_size=16, num_workers=96, persistent_workers=True)
Versions

torchdata==0.6.1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the on_disk_cache entry point and reproduce the issue with the provided DataLoader configuration using multiple workers. Trace the folder-creation path and verify that concurrent workers can create the folder structure and continue without FileExistsError; the supplied pipeline should complete successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.