microsoft / microsoft/winml-cli

Add winml eval support for audio-classification

Open
#1,175 1 comment 0 reactions 1 assignee View on GitHub

@DingmaomaoBJTU is already working on this.

Since Jul 27, 2026.

enhancement model-scale-by-skill P2 triaged
Dominant language
Python
Stars
40
Forks
11
Avg merge
1d 8h
Merged PRs (30d)
50

Description

winml eval currently rejects the audio-classification task before dataset selection, even when winml inspect and winml build resolve/build a standard audio-classification model.

Repro from facebook/mms-lid-256 on current main:

uv run winml eval --schema --task audio-classification
uv run winml eval -m temp\facebook_mms-lid-256_tester\cpu\fp32\model.onnx --model-id facebook/mms-lid-256 --task audio-classification --ep cpu --device cpu

Observed:

Error: Task 'audio-classification' is not supported by `winml eval`.
Error: Evaluation failed: Task 'audio-classification' is not supported.

Expected: an audio-classification eval schema/adapter with dataset preprocessing for raw audio or feature-extractor-normalized waveform input and a task metric such as accuracy.

This blocks Goal-L3 validation for standard Wav2Vec2ForSequenceClassification audio classifiers such as facebook/mms-lid-256.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.