Does anyone download the dataset successfully?
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13.7k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
I'm trying to download the Objaverse-sketchfab part of the released TRELLIS-500K, however, the following error always occurs every several seconds (~20s) of downloading...
Processing 168307 objects...
2025-04-21 19:41:25.977 | INFO | objaverse.xl.sketchfab:download_objects:508 - Found 7329 objects already downloaded
2025-04-21 19:41:28.661 | INFO | objaverse.xl.sketchfab:download_objects:529 - Downloading 160978 new objects across 384 processes
1%|▎ | 1391/160978 [00:20<39:43, 66.96it/s]
Traceback (most recent call last):
File "/workspace/TRELLIS/dataset_toolkits/download.py", line 52, in <module>
downloaded = dataset_utils.download(metadata, **opt)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TRELLIS/dataset_toolkits/datasets/ObjaverseXL.py", line 33, in download
file_paths = oxl.download_objects(
^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/trellis/lib/python3.11/site-packages/objaverse/xl/__init__.py", line 147, in download_objects
source_downloads = downloaders[source].download_objects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/trellis/lib/python3.11/site-packages/objaverse/xl/sketchfab.py", line 551, in download_objects
new_object_downloads = list(
^^^^^
File "/opt/conda/envs/trellis/lib/python3.11/site-packages/tqdm/std.py", line 1181, in __iter__
for obj in iterable:
File "/opt/conda/envs/trellis/lib/python3.11/multiprocessing/pool.py", line 873, in next
raise value
multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7f8e714b3bd0>'. Reason: 'TypeError("cannot pickle '_io.BufferedReader' object")'
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with dataset_toolkits/download.py and dataset_toolkits/datasets/ObjaverseXL.py, then inspect objaverse.xl.sketchfab.download_objects in the traceback. Reproduce the TRELLIS-500K Objaverse-sketchfab download and investigate the multiprocessing pickle failure. Done means the download completes without the reported MaybeEncodingError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100