microsoft / microsoft/onnxruntime-genai

[Feature request]: Add C++ multimodal benchmark (VLM perf testing without Python)

Open
#2,373 0 comments 1 reaction 2 assignees Claimed by @kunal-vaishnavi View on GitHub
platform:mobile
Dominant language
C++
Stars
1.1k
Forks
354
Avg merge
2d 16h
Merged PRs (30d)
85

Description

model_benchmark (C++) only benchmarks text-only decoder models via OgaTokenizer + AppendTokenSequences. There is no C++ equivalent of benchmark/python/benchmark_multimodal.py for vision-language and other multimodal models.

This gap matters for environments where Python is unavailable or undesirable (Android, embedded, CI perf gates, Foundry Local–style deployments) but multimodal models still need end-to-end perf measurement (vision prefill + text decode).

Any plan to support this feature? Thanks!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.