facebookresearch / facebookresearch/fairseq2
DataPipeline execution profiling
Open
question
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 144
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 1
Description
How could one to see the overall performance iterations of a complex DataPipeline and identify potential bottlenecks ?
Ideally we would like to have several time series informations (like CPU/Memory usage) per any single stage in DataPipeline.
Maybe something similar to https://www.tensorflow.org/guide/data_performance_analysis .
Contributor guide
Assessment
This issue has not been assessed yet.