meta-pytorch / meta-pytorch/data
torchdata not compatible with torch 2.3.0
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 179
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 2
Description
🐛 Describe the bug
Describe the bug
torchdata does not work with torch 2.3.0 because DILL_AVAILABLE is not available where expected:
Python 3.12.2 (main, Feb 6 2024, 20:19:44) [Clang 15.0.0 (clang-1500.1.0.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import torchdata
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/private/tmp/test/lib/python3.12/site-packages/torchdata/__init__.py", line 9, in <module>
from . import datapipes
File "/private/tmp/test/lib/python3.12/site-packages/torchdata/datapipes/__init__.py", line 9, in <module>
from . import iter, map, utils
File "/private/tmp/test/lib/python3.12/site-packages/torchdata/datapipes/iter/__init__.py", line 79, in <module>
from torchdata.datapipes.iter.util.cacheholder import (
File "/private/tmp/test/lib/python3.12/site-packages/torchdata/datapipes/iter/util/cacheholder.py", line 24, in <module>
from torch.utils.data.datapipes.utils.common import _check_unpickable_fn, DILL_AVAILABLE
ImportError: cannot import name 'DILL_AVAILABLE' from 'torch.utils.data.datapipes.utils.common' (/private/tmp/test/lib/python3.12/site-packages/torch/utils/data/datapipes/utils/common.py)
>>>
This was fixed in https://github.com/pytorch/pytorch/pull/122616 but the fix did not make it into the torch 2.3.0 release
https://github.com/pytorch/pytorch/blob/v2.3.0/torch/utils/data/datapipes/utils/common.py
I would assume 2.3.1 will be ok because the fix is in main. For now though, a clean install of torchdata is broken.
I would recommend adding a torch>=2,!=2.3.0 to setup.py
To Reproduce
pip install torchdata
import torchdata
Expected behavior
successful import
Environment
fresh install
Versions
Collecting environment information...
PyTorch version: 2.3.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A
OS: macOS 14.4.1 (arm64)
GCC version: Could not collect
Clang version: 15.0.0 (clang-1500.3.9.4)
CMake version: version 3.27.1
Libc version: N/A
Python version: 3.12.2 (main, Feb 6 2024, 20:19:44) [Clang 15.0.0 (clang-1500.1.0.2.5)] (64-bit runtime)
Python platform: macOS-14.4.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
CPU:
Apple M1 Max
Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] torch==2.3.0
[pip3] torchdata==0.7.1
[conda] Could not collect
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with setup.py and the torchdata import path shown in the traceback, then reproduce the failure with torch 2.3.0 using pip install torchdata and import torchdata. Update the dependency constraints to exclude the incompatible torch release, and verify that installation and import succeed with supported versions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- data, machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100