gpt2 / text-generation: all models pass wmk perf

Summary

openai-community/gpt2 export completes successfully but wmk perf fails because WinMLModelForCausalLM is not implemented — the inference wrapper for text-generation falls back to WinMLModelForGenericTask which does not handle autoregressive decoding correctly.

Eval Results (2026-03-11)

1/1 gpt2 model fails.

Note: All Qwen2/Qwen3 text-generation models fail for the same root cause (tracked in issue #432).

Root Cause

modelkit/models/winml/__init__.py:37 maps text-generation → WinMLModelForCausalLM, but WinMLModelForCausalLM is not in the _import_winml_class() map. The fallback to WinMLModelForGenericTask does not provide autoregressive decoding, so the benchmark either produces wrong output or crashes.

Unlike Qwen2/Qwen3, GPT-2 is small enough that the ONNX export succeeds (no protobuf size issue) — the only blocker is the missing inference class.

Current State

• modelkit/models/winml/__init__.py:37 — "text-generation": "WinMLModelForCausalLM" (mapped but not implemented)
• _import_winml_class() (modelkit/models/winml/__init__.py:56-91) — no entry for WinMLModelForCausalLM
• GPT-2 export path works (confirmed by stdout showing completed build stages)

Desired State

openai-community/gpt2 / text-generation passes wmk perf.

Acceptance Criteria

• openai-community/gpt2 passes wmk perf
• WinMLModelForCausalLM implemented as a proper inference wrapper
• Fix also unblocks Qwen2/Qwen3 smaller models once their ONNX size issue is resolved (#432)
• Fix is universal — works for any causal LM, not just GPT-2 (CLAUDE.md Cardinal Rule #1)
• uv run pytest tests/ passes (CLAUDE.md Cardinal Rule #3)

Technical Notes

• WinMLModelForCausalLM needs to: load ONNX session, generate input_ids, run a single forward pass (not full generation loop for benchmarking), return logits
• For wmk perf purposes, a single forward pass with fixed input_ids is sufficient — no need to implement full greedy/beam decode
• Check modelkit/models/winml/sequence_classification.py as implementation pattern
• GPT-2 ONNX inputs: input_ids (int64, shape [batch, seq]) + optionally attention_mask — confirm via wmk inspect -m openai-community/gpt2

Related Files

• modelkit/models/winml/__init__.py:37,56-91 — task mapping and class import
• modelkit/models/winml/sequence_classification.py — implementation pattern
• modelkit/models/winml/image_classification.py — another pattern
• eval_results/2026-03-11/models/openai-community__gpt2__text-generation/result.json

References

• Related: issue #432 (Qwen2/Qwen3 text-generation) — same missing class

Start with modelkit/models/winml/init.py, especially _import_winml_class(), and compare modelkit/models/winml/sequence_classification.py and image_classification.py. Run wmk inspect -m openai-community/gpt2 to confirm the ONNX inputs, then use the existing evaluation result as a baseline. Done means GPT-2 passes wmk perf, the causal-LM wrapper is mapped universally, and uv run pytest tests/ passes.