meta-pytorch / meta-pytorch/data
`_DataPipeSerializationWrapper` doesn't work with multiprocessing Queue
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 179
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 2
Description
🐛 Describe the bug
After https://github.com/pytorch/data/pull/919 is landed, a hanging problem happens on MacOS or Windows, where spawn is used to create subprocesses by default. See: https://github.com/pytorch/data/actions/runs/3794926183
I was able to mitigate the issue by removing the SeializationWrapper from https://github.com/pytorch/data/blob/e15e1453967ce2f25f6fcd2838caadfd0e2fa811/torchdata/dataloader2/dataloader2.py#L112
And, the reason that the SerializationWrapper doesn't work is multiprocessing.Queue is attached to a DataPipe and sent to subprocesses. Even though I am able to solve my hanging problem in a different way, it's better to solve this problem directly via SerializationWrapper.
The following should be a minimum repro example
ctx = mp.get_context("spawn")
q = ctx.Queue()
dp = IterableWrapper(list(range(10)))
# Attach a Queue
dp.q = q
dl = DataLoader2(dp, reading_service=PrototypeMultiProcessingReadingService(2, "spawn"))
for d in dl:
pass
cc: @NivekT
Versions
main
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with torchdata/dataloader2/dataloader2.py around line 112 and the referenced SerializationWrapper in torch/utils/data/datapipes/datapipe.py. Run the provided spawn-based multiprocessing Queue reproducer on macOS or Windows and inspect the linked PR 919 changes. Done means the wrapper no longer causes the DataLoader2 process to hang when a DataPipe has an attached Queue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100