facebookresearch / facebookresearch/fairseq2
Audio Data Loader Example?
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 144
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 1
Description
Hi, I'd like to use FairSeq2 for my audio models. Is there an example of an audio data loader in the works? That would be super helpful.
What I need in the dataloader:
1. I have lots of audio that I'd like to load. This is currently in mp3, but I've been cutting chunks out as tuples of (audio: Wav, labels: Json).
2. This data is hosted on R2 and I'd like the loader to stream it via an endpoint_url.
3. I also would like for this to save to local cache as it's a big dataset.
4. Distribute amongst cpu workers.
I can accommodate other data saving formats as needed, e.g. a flat zip, but the most important thing is that it's fast. I'm right now I/O bottlenecked.
Thanks!
Contributor guide
Assessment
This issue has not been assessed yet.