microsoft / microsoft/winml-cli
[Task] sentence-similarity EP coverage investigation
@ssss141414 is already working on this.
Since Apr 9, 2026.
- Dominant language
- Python
- Stars
- 40
- Forks
- 11
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 50
Description
Overview
Sentence similarity models produce dense sentence embeddings optimized for semantic textual similarity tasks (STS). These include Sentence-Transformers and BAAI/BGE variants. All 10 models pass on at least one EP.
This issue focuses on the 3 models with incomplete EP coverage — all exhibiting the same VitisAI failure pattern. The affected models are the multilingual MiniLM and two smaller BGE variants, suggesting the failure may be related to model size or multilingual tokenizer handling on VitisAI.
Agent Scenarios
- Semantic deduplication agent: identify and merge duplicate or near-duplicate records (support tickets, product listings, knowledge base articles) by comparing sentence embeddings
- Cross-lingual search agent: retrieve semantically similar content across languages using multilingual models (paraphrase-multilingual-*) for global enterprise search
- RAG similarity scoring agent: compute relevance between retrieved passages and a query to filter or rerank before LLM generation
EP Coverage Status
| Model | QNN | OV | VitisAI |
|---|---|---|---|
| sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 | PASS | PASS | FAIL |
| BAAI/bge-small-en-v1.5 | PASS | PASS | FAIL |
| BAAI/bge-base-en-v1.5 | PASS | PASS | FAIL |
Reference: all-MiniLM-L6-v2, all-mpnet-base-v2, bge-m3, paraphrase-multilingual-mpnet-base-v2, bge-large-en-v1.5, multi-qa-mpnet-base-dot-v1, multilingual-e5-large already pass all 3 EPs.
Acceptance Criteria
- sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 passes all 3 EPs (investigate VitisAI failure)
- BAAI/bge-small-en-v1.5 passes all 3 EPs (investigate VitisAI failure)
- BAAI/bge-base-en-v1.5 passes all 3 EPs (investigate VitisAI failure)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.