meta-pytorch / meta-pytorch/data
Functional API document auto generation
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 179
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 2
Description
🚀 The feature
Currently, we rely on the inline doc for each DataPipe to indicate the functional API. Considering torchdata should have already been built when doc need to be generated, we should be able to rely on the following dictionaries to figure out the functional API with corresponding DataPipe.
- IterDataPipe: https://github.com/pytorch/pytorch/blob/b30c027abf6560dafb88d67ebd446851a5729651/torch/utils/data/datapipes/datapipe.py#L68
- MapDataPipe: https://github.com/pytorch/pytorch/blob/ea8a0184b76088c477c7cef4fe36d683d57fd880/torch/utils/data/datapipes/datapipe.py#L194
Motivation, pitch
Make the doc-gen more intelligent to reduce the amount of work when contributors implement DataPipe and the extra care for reviewers/
Alternatives
No response
Additional context
WDYT @NivekT
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the IterDataPipe and MapDataPipe functional API dictionaries linked in the issue, then locate the repository's doc-generation entry point. Define how those dictionaries should map DataPipe implementations to generated documentation, and confirm that generation produces the expected functional API documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100