microsoft / microsoft/winml-cli

[Task] audio-to-audio (vocoder) model support

Open
#294 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

model / task scale P2 triaged
Dominant language
Python
Stars
40
Forks
11
Avg merge
1d 8h
Merged PRs (30d)
50

Description

Overview

Audio-to-audio vocoder models convert intermediate acoustic representations (mel spectrograms, acoustic features) into raw waveforms. They are a critical component of TTS (text-to-speech) and voice conversion pipelines. Target: top 2 vocoder models by HuggingFace downloads (>2k), e.g. HiFi-GAN, BigVGAN, or similar.

Agent Scenarios

  • TTS synthesis agent: serve as the waveform decoder stage in a full TTS pipeline (acoustic model → vocoder → playback), enabling high-quality on-device speech synthesis
  • Voice conversion agent: transform the timbre of a source speaker's waveform to match a target speaker's voice profile
  • Accessibility agent: convert synthesized mel spectrograms to natural-sounding speech for screen readers or communication aids running locally on NPU
  • Podcast / audio production agent: apply a neural vocoder as a post-processing stage to enhance or re-synthesize degraded audio

ModelKit Integration

Models must pass the full wmk pipeline on all EPs:

wmk config → wmk build (ONNX export) → wmk perf → wmk eval

Acceptance Criteria

  • Identify top 2 vocoder models (>2k downloads) and add to model list
  • All selected models pass wmk perf on all EPs

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the model list and the existing wmk pipeline integration, then review how models are handled by wmk config, wmk build, wmk perf, and wmk eval. Identify two vocoder models with more than 2k HuggingFace downloads, add them to the model list, and verify the full pipeline and wmk perf results on all EPs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.