qwen2 + qwen3 / text-generation: all models pass wmk perf

Summary

All Qwen2 and Qwen3 text-generation models fail. Smaller models hit Error parsing message with type 'onnx.ModelProto' (protobuf 2GB limit), while larger models timeout. This blocks the entire Qwen LLM family.

Eval Results (2026-03-11)

8/8 models fail — 0 pass.

Root Cause

1. ONNX ModelProto size: LLMs generate ONNX graphs >2GB, hitting protobuf's message size limit. Requires external data format (save_as_external_data=True).
2. Compilation/runtime crash: Qwen2.5-7B exits with code 3221225477 (Windows access violation / memory error) — likely OOM during compilation.
3. Tiny model perf failure: tiny-Qwen2ForCausalLM-2.5 export completes but perf fails — likely input generation issue for text-generation (autoregressive models need different input handling).

Current State

• No qwen2.py or qwen3.py in modelkit/models/hf/
• No text-generation WinML inference class: modelkit/models/winml/__init__.py:37 maps text-generation → WinMLModelForCausalLM but this class is not implemented (falls back to WinMLModelForGenericTask)
• openai-community/gpt2 also fails text-generation (gpt2 model_type) with export completing but perf failing — same missing CausalLM inference class

Desired State

Qwen2 and Qwen3 models pass wmk perf for text-generation, at least for smaller sizes (0.5B–3B).

Acceptance Criteria

• Qwen/Qwen2.5-0.5B-Instruct, Qwen2.5-1.5B-Instruct, Qwen2.5-3B-Instruct pass wmk perf
• Qwen/Qwen3-0.6B, Qwen3-1.7B pass wmk perf
• trl-internal-testing/tiny-Qwen2ForCausalLM-2.5 passes wmk perf
• External data format enabled for LLMs exceeding ONNX size threshold (CLAUDE.md Cardinal Rule #1 — size-based, not model-name-based)
• WinMLModelForCausalLM inference wrapper implemented
• uv run pytest tests/ passes (CLAUDE.md Cardinal Rule #3)
• Eval re-run confirms ≥5/8 pass (7B+ size may remain gated by hardware memory)

Technical Notes

• ONNX size: Enable use_external_data_format=True for large model exports — coordinate with xlm-roberta issue which has the same fix
• CausalLM inference: WinMLModelForCausalLM is mapped in TASK_TO_WINML_CLASS but not implemented in modelkit/models/winml/. Need to implement at minimum a greedy-decode single-pass wrapper
• Input generation for text-generation: Autoregressive models need input_ids + attention_mask — check modelkit/core/model_input_generator.py for correct handling
• Large model OOM: 7B models may need memory-mapped loading or INT4 quantization path — out of scope for this issue unless hardware supports it
• Same onnx.ModelProto error affects xlm-roberta and laion/CLIP-ViT-H-14 — a single shared fix for external data format covers all

Related Files

• modelkit/models/winml/__init__.py:37 — WinMLModelForCausalLM mapping (not implemented)
• modelkit/export/config.py — WinMLExportConfig — check use_external_data_format field
• modelkit/core/model_input_generator.py — input generation
• modelkit/models/hf/bert.py — pattern for adding model-specific ONNX config
• eval_results/2026-03-11/models/ — Qwen result.json files

References

• CLAUDE.md Cardinal Rule #1: No hardcoded model type strings — use size-based detection for external data format
• CLAUDE.md Cardinal Rule #3: Run uv run pytest tests/ after implementation

Start with modelkit/models/winml/init.py, modelkit/export/config.py, and modelkit/core/model_input_generator.py to trace the text-generation mapping, external-data setting, and inputs. Compare model-specific export patterns in modelkit/models/hf/bert.py and run the existing tests before changing behavior. Done means the listed smaller Qwen models and tiny model pass wmk perf, with external data enabled by size and uv run pytest tests/ passing.