facebookresearch / facebookresearch/fairseq2

HuBERT pre-training recipe

Open
#1,498 0 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
1.1k
Forks
144
Avg merge
4d 1h
Merged PRs (30d)
1

Description

**Is your feature request related to a problem? Please describe:**
I have relied on fairseq for pre-training Wav2Vec2 and HuBERT models in the past; I see that fairseq2 includes [Wav2Vec2](https://github.com/facebookresearch/fairseq2/tree/main/recipes/wav2vec2) but seemingly lacks functionality for HuBERT at the moment.

**Describe the solution you would like:**
Include functionality for pre-training HuBERT in fairseq2, as it was available in [deprecated fairseq](https://github.com/facebookresearch/fairseq/tree/main/examples/hubert).

**Describe the alternatives you have considered:**
I want to be able to pre-train both Wav2Vec2 and HuBERT models within the same toolkit. Alternatively to fairseq2 I would consider switching to huggingface or PyTorch-native training, thus sadly missing out on fairseq2.

**Additional Context:**
HuBERT pre-training differs minimally from Wav2Vec2, only in pre-training objective and use of iterative refinement. It is still one of the most common architectures used for speech SSL and HuBERT models often outperform Wav2Vec2 on downstream tasks.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.