meta-pytorch/data

View on GitHub

A PyTorch repo for data loading and utilities to be shared by the PyTorch domain libraries.

Stars: 1.3k
Forks: 179
Open beginner issues: 0
Indexed issues: 221
Avg merge: 6d 1h
Merged PRs (30d): 2

Dominant language: Python
License: BSD-3-Clause
Last GitHub push: Sep 16, 2026
Latest indexed: Sep 20, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: good first issue help wanted

Beginner friendly Everything

Open All

221 open issues indexed

Loader checkpoint restoration restarts before a None item Open

Difficulty 2/5 1-3 hours Newbie friendliness 78/100

meta-pytorch/data#1551 · 1 comment · Opened Aug 23, 2026
Docs setup needs to be upgraded to use PyTorch Sphinx Theme2 Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

meta-pytorch/data#1533 · Opened Mar 13, 2026
Torch CPU index is missing torchdata==0.11.0 Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

meta-pytorch/data#1532 · Opened Mar 7, 2026
[feature-request] Block data that is flagged by a CC "no-AI-training" license from being ingested into a DataLoader Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

meta-pytorch/data#1509 · 2 comments · Opened Oct 20, 2025
[feature request] Upstream to core PyTorch `StatefulDataLoader` and `StatefulDistributedSampler` Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

meta-pytorch/data#1508 · 2 comments · Opened Sep 17, 2025
Add a Stack node for ensembling Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

meta-pytorch/data#1507 · 9 comments · 1 reaction · Opened Aug 21, 2025
v0.12.0 (or 0.11.1?) release timeline Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

meta-pytorch/data#1506 · 3 comments · 6 reactions · Opened Aug 16, 2025
Add an example using a library such as Hugging Face datasets Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

meta-pytorch/data#1504 · 2 comments · Opened Aug 8, 2025
`enumerate()` index doesn't reflect StatefulDataLoader's actual position after loading state Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

meta-pytorch/data#1503 · 4 comments · Opened Aug 7, 2025
Async Checkpointing with DCP and Stateful Dataloader Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

meta-pytorch/data#1502 · 2 comments · Opened Aug 6, 2025
ParallelMapper missing a `worker_init_fn` Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

meta-pytorch/data#1494 · 1 comment · Opened Jun 4, 2025
online hard-example mining under Multi-GPU (DDP) Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

meta-pytorch/data#1493 · Opened Jun 4, 2025
Implement a Cache node Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

meta-pytorch/data#1489 · 1 comment · 3 reactions · Opened May 15, 2025
Expose pin_memory callable for custom objects Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

meta-pytorch/data#1481 · 1 comment · 1 reaction · Opened May 2, 2025
torchdata or torchdata-contrib? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

meta-pytorch/data#1471 · 2 comments · 2 reactions · Opened Apr 6, 2025
`IterableWrapper` fails on an SQLite cursor Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

meta-pytorch/data#1470 · 3 comments · Opened Mar 31, 2025
Enable GPU unit tests for nodes Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

meta-pytorch/data#1467 · Opened Mar 20, 2025
Explore lifting the restriction on `max_concurrent` Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

meta-pytorch/data#1462 · Opened Mar 9, 2025
Discussion: DCP APIs and broader contracts for rescalability Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

meta-pytorch/data#1456 · 2 comments · 3 reactions · Opened Feb 25, 2025
Open for contribution on utility nodes like `Filter`, `Shuffler`, `Header`, `Cycler`? Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

meta-pytorch/data#1452 · 1 comment · 3 reactions · Opened Feb 25, 2025
[Discussion] `def Mapper` may not be ideal Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

meta-pytorch/data#1444 · 1 comment · Opened Feb 16, 2025
torch.node datawriter Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

meta-pytorch/data#1418 · Opened Dec 27, 2024
[RFC] Remove the need for super().__init__() call in BaseNode subclasses to better support dataclasses Open

nodes

Difficulty 5/5 Over a week Newbie friendliness 30/100

meta-pytorch/data#1411 · Opened Dec 18, 2024
[Nodes] Add a ToDevice node, or combine with pin memory Open

nodes

Difficulty 5/5 Over a week Newbie friendliness 25/100

meta-pytorch/data#1407 · 2 comments · 1 reaction · Opened Dec 13, 2024
[Nodes] ParallelMap state is not guraranteed with in_order=False Open

nodes

Difficulty 4/5 3-5 days Newbie friendliness 38/100

meta-pytorch/data#1405 · Opened Dec 13, 2024
[Nodes] Drop-in Replacement for DataLoader and StatefulDataLoader Open

nodes

Difficulty 5/5 Over a week Newbie friendliness 25/100

meta-pytorch/data#1389 · Opened Dec 9, 2024
[Nodes] Ensure get_worker_info works correctly in ParallelMapper Open

nodes

Difficulty 5/5 Over a week Newbie friendliness 30/100

meta-pytorch/data#1388 · Opened Dec 9, 2024
[Stateful DataLoader] CI takes too long for Mac and WIndows Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

meta-pytorch/data#1338 · 1 comment · Opened Oct 16, 2024
[Stateful DataLoader] add utilities to inspect/modify dataloader state Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

meta-pytorch/data#1337 · Opened Oct 15, 2024
[RFC] Polylithic: Enabling multi-threaded DataLoading through non-monolithic parallelism Open

nodes

Difficulty 5/5 Over a week Newbie friendliness 20/100

meta-pytorch/data#1334 · 11 comments · 15 reactions · Opened Oct 10, 2024
early stop worker got Exception Error Open

datapipes

Difficulty 4/5 3-5 days Newbie friendliness 35/100

meta-pytorch/data#1310 · 2 comments · Opened Aug 5, 2024
best practice for `snapshot_every_n_steps` Open

documentation

meta-pytorch/data#1283 · 5 comments · 1 assignee · Opened Jul 7, 2024
Enable Append Mode in SaverIterDataPipe Open

datapipes wontfix

Difficulty 3/5 1-2 days Newbie friendliness 35/100

meta-pytorch/data#1270 · 1 comment · Opened Jun 10, 2024
State_dict on dataset seems to be called more often than expected Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

meta-pytorch/data#1268 · 2 comments · 1 reaction · Opened Jun 10, 2024
GDriveReaderDataPipe complains "using a sharing/viewing link instead of a download link" Open

datapipes wontfix

Difficulty 3/5 1-2 days Newbie friendliness 35/100

meta-pytorch/data#1265 · Opened May 31, 2024
torchdata not compatible with torch 2.3.0 Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

meta-pytorch/data#1244 · 3 comments · 21 reactions · Opened Apr 25, 2024
Move to removesuffix string method after python 3.8 support is dropped Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

meta-pytorch/data#1240 · Opened Mar 27, 2024
DataLoader2 with multiprocess raise exception: Can not request next item while we are still waiting response for previous request Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

meta-pytorch/data#1228 · 4 comments · Opened Mar 12, 2024
Dataloader is slow with iterdatapipes and shuffle that has large in-memory fields (because traverse_dps is slow) Open

module: dataloader triaged

Difficulty 4/5 3-5 days Newbie friendliness 35/100

meta-pytorch/data#1226 · 3 comments · Opened Feb 9, 2024
`v2.1.2+cu118` and `v2.1.1+cu118` run into torchdata `ImportError: libssl.so.3: cannot open shared object file: No such file or directory`, that `v2.1.0+cu118` doesn't have an issue with Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

meta-pytorch/data#1220 · 1 comment · 1 reaction · Opened Jan 11, 2024
Iterating a data pipe, created with random split, ends in error as the code tries to iterate past the data pipe lenght Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

meta-pytorch/data#1218 · Opened Jan 4, 2024
S3FileLoaderIterDataPipe buffer_size Open

Difficulty 1/5 Under an hour Newbie friendliness 45/100

meta-pytorch/data#1215 · Opened Nov 17, 2023
Loading `.tfrecords` files that require a deserialization method Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

meta-pytorch/data#1201 · 1 comment · 3 reactions · Opened Sep 26, 2023
Calling __iter__ twice on DataLoader2 causes hang with MPRS Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

meta-pytorch/data#1198 · 2 comments · Opened Jul 21, 2023
Future of torchdata and dataloading Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

meta-pytorch/data#1196 · 63 comments · 58 reactions · Opened Jul 17, 2023
An iterator that can stream over stdin Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

meta-pytorch/data#1193 · Opened Jul 4, 2023
Dataloader2 with FullSyncIterDataPipe throws error during initilization Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

meta-pytorch/data#1190 · 3 comments · Opened Jun 19, 2023
FileExistsError when using `on_disk_cache` and multiple workers Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

meta-pytorch/data#1186 · 1 comment · Opened Jun 16, 2023
DataLoader2 Memory Behavior is very strange on Epoch Resets Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

meta-pytorch/data#1185 · 9 comments · Opened Jun 14, 2023
Roadmap for mixed chain of multithread and multiprocessing pipelines? Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

meta-pytorch/data#1184 · 2 comments · Opened Jun 14, 2023

meta-pytorch/data

Get new issues in your inbox