huggingface / huggingface/transformers

Add TUNet

Open
#15,763 4 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

New model
Dominant language
Python
Stars
166k
Forks
34.6k
Avg merge
3d 9h
Merged PRs (30d)
281

Description

# 🌟 New model addition

The TUNet model for audio superresolution

## Model description

[TUNet: A Block-online Bandwidth Extension Model based on Transformers and Self-supervised Pretraining](https://arxiv.org/abs/2110.13492v3) is a paper by Viet-Anh Nguyen, Anh H. T. Nguyen, and Andy W. H. Khong, introducing a model for audio superresolution based on transformers.

Audio superresolution allows for the upsampling of audio with minimal loss of quality. This is very useful for ASR tasks that require audio to be resampled during preprocessing, which has a big impact on transcriptions depending on the native sample rate.

## Open source status

* [x] the model implementation is available: [The official repo](https://github.com/NXTProduct/TUNet)
* [x] the model weights are available: [ONNX weights](https://github.com/NXTProduct/TUNet/tree/master/lightning_logs)
* [x] who are the authors: Viet-Anh Nguyen, Anh H. T. Nguyen, and Andy W. H. Khong

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the TUNet paper and the official TUNet repository linked in the issue, including its available ONNX weights. Compare the model and weights with existing Transformers audio model integrations; done means TUNet is implemented and its available weights are integrated and validated for audio superresolution.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.