meta-pytorch / meta-pytorch/data
[DataPipe] DataPipe Deprecation Tracker
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 179
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 2
Description
We have a number of DataPipes that are being deprecated. Our general policy is that we first mark the DataPipe as deprecated with a warning, and wait at least one release cycle (~3 months) before removing it. Note that some DataPipes will be removed from the PyTorch Core library but will remain in TorchData, and some others are renamed.
Status Types:
- Deprecated - marked as deprecated with a warning
- Removed - removed from repository
DataLoader2 Tracker
| Name | Deprecation Date | Status | Earliest Removal Version |
|---|---|---|---|
PrototypeMultiProcessingReadingService -> MultiProcessingReadingService |
0.6 | Deprecated | 0.8 |
IterDataPipe Tracker
| Name | Functional API | Module | Deprecation Date | Status | Earliest Removal Version |
|---|---|---|---|---|---|
| BucketBatcher | NA | Core | Sep 30th, 2021 | Removed (moved to TorchData) | |
| HTTPReader | NA | Core | Sep 30th, 2021 | Removed (moved to TorchData) | |
| LineReader | NA | Core | Sep 30th, 2021 | Removed (moved to TorchData) | |
| TarArchiveReader | NA | Core | Sep 30th, 2021 | Removed (moved to TorchData) | |
| ZipArchiveReader | NA | Core | Sep 30th, 2021 | Removed (moved to TorchData) | |
| FileLoader | NA | Core | Jan 5th, 2022 | Removed (use FileOpener) | 1.13 (Sept 2022) |
| FileLoader | NA | Data | Jan 5th, 2022 | Removed (use FileOpener) | |
| IoPathFileLoader | load_file_by_iopath | Data | Jan 5th, 2022 | Removed (use IoPathFileOpener) | |
| RoutedDecoder | routed_decode | Core | Jan 10th, 2022 | Deprecated | 1.13 (Sept 2022) |
| TarArchiveReader | read_from_tar | Data | Feb 22th, 2022 | Removed (use TarArchiveLoader) | 0.5 (Sept 2022) |
| XzFileReader | read_from_xz | Data | Feb 22th, 2022 | Removed (use XzFileLoader) | 0.5 (Sept 2022) |
| ZipArchiveReader | read_from_zip | Data | Feb 22th, 2022 | Removed (use ZipArchiveLoader) | 0.5 (Sept 2022) |
| Filter | filter | Core | 1.12 | Removed argument (drop_empty_batches) | 2.0 (Nov 2022) |
| FSSpecFileOpener | open_files_by_fsspec | Data | 0.4 | open_file_by_fsspec is Removed |
0.6 (Nov 2022) |
| IoPathFileOpener | open_files_by_fsspec | Data | 0.4 | open_file_by_iopath is Removed |
0.6 (Nov 2022) |
MapDataPipe Tracker
Nothing for now
cc: @ejguan @VitalyFedyunin @NivekT
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the DataLoader2 and IterDataPipe tracker tables in this issue and compare each listed DataPipe with the repository's current deprecation and removal state. Update the relevant status, replacement name, or removal version while preserving the documented status types and release-cycle policy; done means the tracker reflects the current state.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100