siglip_vision_model / image-feature-extraction: all models pass wmk perf

Summary

hi-wesley/gemma3-vision-encoder (model_type: siglip_vision_model) fails with number of output names provided (2) exceeded number of outputs (1). The ONNX config registers 2 output names but the model only produces 1 output tensor.

Eval Results (2026-03-11)

1/1 siglip_vision_model fails.

Root Cause

The ONNX export config for siglip_vision_model defines 2 output names (likely image_embeds + last_hidden_state) but the gemma3-vision-encoder only exposes 1 output (last_hidden_state or image_embeds). This is a mismatch between the registered ONNX config outputs and the actual model architecture for this specific variant.

Note: hi-wesley/gemma3-vision-encoder is the SigLIP vision encoder extracted from Gemma3's multimodal architecture — it may not produce the same outputs as the standard SigLIP vision model.

Current State

• No siglip.py in modelkit/models/hf/
• The ONNX output config for siglip_vision_model likely comes from Optimum defaults
• Standard SigLIP vision model produces image_embeds + last_hidden_state; Gemma3's extracted encoder may only produce last_hidden_state

Desired State

hi-wesley/gemma3-vision-encoder passes wmk perf.

Acceptance Criteria

• hi-wesley/gemma3-vision-encoder passes wmk perf
• Fix is universal — output count detection from actual model, not hardcoded (CLAUDE.md Cardinal Rule #1)
• uv run pytest tests/ passes (CLAUDE.md Cardinal Rule #3)

Technical Notes

• Add a siglip_vision_model ONNX config override that dynamically determines outputs from the model's actual config.json rather than assuming a fixed set
• Or: register two separate configs — one for standard SigLIP vision model (2 outputs) and one for Gemma3-extracted encoder (1 output), distinguished by checking model config attributes
• The CLIPVisionModelIOConfig in modelkit/models/hf/clip.py is a close reference — SigLIP vision model follows a similar pattern

Related Files

• modelkit/models/hf/clip.py:106-131 — CLIPVisionModelIOConfig (very similar pattern)
• modelkit/export/io.py — register_onnx_overwrite()
• eval_results/2026-03-11/models/hi-wesley__gemma3-vision-encoder__image-feature-extraction/result.json

Start with modelkit/models/hf/clip.py:106-131 for the related CLIPVisionModelIOConfig, then inspect modelkit/export/io.py and the reported result.json. Verify how the siglip_vision_model output configuration is registered and compare it with the Gemma3 encoder's actual outputs. Done means the model passes wmk perf and uv run pytest tests/ passes.