microsoft / microsoft/winml-cli

[Task] question-answering EP coverage investigation

Open
#300 0 comments 0 reactions 1 assignee View on GitHub

@ssss141414 is already working on this.

Since Apr 9, 2026.

model / task scale P2 triaged
Dominant language
Python
Stars
40
Forks
11
Avg merge
1d 8h
Merged PRs (30d)
50

Description

Overview

Extractive QA models (BERT, RoBERTa, DeBERTa, ELECTRA) identify answer spans from a context passage. 9 of 10 models already pass all 3 EPs. This issue targets the single remaining model with incomplete coverage.

tau/splinter-base uses the SPLINTER architecture (span-level representation) which differs structurally from standard BERT-family models — the VitisAI failure may be related to its span selection head or non-standard attention patterns.

Agent Scenarios

  • See #270 for full QA agent scenarios (document Q&A, customer support, research assistant, on-device copilot)

EP Coverage Status

Model QNN OV VitisAI
tau/splinter-base PASS PASS FAIL

Reference: all other 9 QA models pass all 3 EPs.

Acceptance Criteria

  • tau/splinter-base passes all 3 EPs (investigate VitisAI failure)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.