microsoft / microsoft/TRELLIS

Does anyone download the dataset successfully?

Open
#248 5 comments 6 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13.7k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

I'm trying to download the Objaverse-sketchfab part of the released TRELLIS-500K, however, the following error always occurs every several seconds (~20s) of downloading...

Processing 168307 objects...
2025-04-21 19:41:25.977 | INFO     | objaverse.xl.sketchfab:download_objects:508 - Found 7329 objects already downloaded
2025-04-21 19:41:28.661 | INFO     | objaverse.xl.sketchfab:download_objects:529 - Downloading 160978 new objects across 384 processes
  1%|▎                                          | 1391/160978 [00:20<39:43, 66.96it/s]
Traceback (most recent call last):
  File "/workspace/TRELLIS/dataset_toolkits/download.py", line 52, in <module>
    downloaded = dataset_utils.download(metadata, **opt)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/TRELLIS/dataset_toolkits/datasets/ObjaverseXL.py", line 33, in download
    file_paths = oxl.download_objects(
                 ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/trellis/lib/python3.11/site-packages/objaverse/xl/__init__.py", line 147, in download_objects
    source_downloads = downloaders[source].download_objects(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/trellis/lib/python3.11/site-packages/objaverse/xl/sketchfab.py", line 551, in download_objects
    new_object_downloads = list(
                           ^^^^^
  File "/opt/conda/envs/trellis/lib/python3.11/site-packages/tqdm/std.py", line 1181, in __iter__
    for obj in iterable:
  File "/opt/conda/envs/trellis/lib/python3.11/multiprocessing/pool.py", line 873, in next
    raise value
multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7f8e714b3bd0>'. Reason: 'TypeError("cannot pickle '_io.BufferedReader' object")'

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with dataset_toolkits/download.py and dataset_toolkits/datasets/ObjaverseXL.py, then inspect objaverse.xl.sketchfab.download_objects in the traceback. Reproduce the TRELLIS-500K Objaverse-sketchfab download and investigate the multiprocessing pickle failure. Done means the download completes without the reported MaybeEncodingError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.