microsoft / microsoft/winml-cli
[Task] question-answering EP coverage investigation
Open
@ssss141414 is already working on this.
Since Apr 9, 2026.
model / task scale
P2
triaged
- Dominant language
- Python
- Stars
- 40
- Forks
- 11
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 50
Description
Overview
Extractive QA models (BERT, RoBERTa, DeBERTa, ELECTRA) identify answer spans from a context passage. 9 of 10 models already pass all 3 EPs. This issue targets the single remaining model with incomplete coverage.
tau/splinter-base uses the SPLINTER architecture (span-level representation) which differs structurally from standard BERT-family models — the VitisAI failure may be related to its span selection head or non-standard attention patterns.
Agent Scenarios
- See #270 for full QA agent scenarios (document Q&A, customer support, research assistant, on-device copilot)
EP Coverage Status
| Model | QNN | OV | VitisAI |
|---|---|---|---|
| tau/splinter-base | PASS | PASS | FAIL |
Reference: all other 9 QA models pass all 3 EPs.
Acceptance Criteria
- tau/splinter-base passes all 3 EPs (investigate VitisAI failure)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.