layoutlm + layoutlmv3 / document-question-answering: all models pass wmk perf

Summary

All LayoutLM and LayoutLMv3 models fail on document-question-answering because the task is not registered in Optimum's TasksManager for these model types.

Eval Results (2026-03-11)

6/6 models fail — 0 pass.

Root Cause

document-question-answering is not registered as a supported ONNX export task for layoutlm and layoutlmv3 model types in Optimum's TasksManager. These models support it at inference time, but the ONNX export path is not configured.

LayoutLM document-question-answering uses input_ids, attention_mask, token_type_ids, bbox (bounding boxes) — the non-standard bbox input means the standard question-answering ONNX config doesn't apply.

Note: zpm/Llama-3.1-PersianQA is misrouted as document-question-answering — it should be question-answering. Not a LayoutLM issue; tracked separately.

Current State

• No layoutlm.py or layoutlmv3.py in modelkit/models/hf/
• document-question-answering task has no ONNX config in Optimum for these model types
• Optimum supports LayoutLM for some tasks — check what's available

Desired State

All 4 LayoutLM/LayoutLMv3 models pass wmk perf.

Acceptance Criteria

• impira/layoutlm-document-qa and impira/layoutlm-invoices pass wmk perf
• rubentito/layoutlmv3-base-mpdocvqa and xhyi/layoutlmv3_docvqa_t11c5000 pass wmk perf
• Fix is universal — no hardcoded model names (CLAUDE.md Cardinal Rule #1)
• uv run pytest tests/ passes (CLAUDE.md Cardinal Rule #3)

Technical Notes

• Register document-question-answering task for layoutlm and layoutlmv3 via @register_onnx_overwrite()
• The ONNX config must include the bbox input tensor (shape: [batch, seq, 4], dtype int64) — this is non-standard and requires a custom LayoutLMIOConfig
• LayoutLMv3 also takes pixel_values in addition to text inputs — multimodal input handling required
• Check Optimum's source for existing LayoutLMOnnxConfig / LayoutLMv3OnnxConfig to see if they can be adapted

Related Files

• modelkit/models/hf/bert.py — pattern for custom ONNX config with custom inputs
• modelkit/export/io.py — register_onnx_overwrite()
• eval_results/2026-03-11/models/impira__layoutlm-document-qa__document-question-answering/result.json
• eval_results/2026-03-11/models/rubentito__layoutlmv3-base-mpdocvqa__document-question-answering/result.json

Start with modelkit/models/hf/bert.py and modelkit/export/io.py, then inspect Optimum's existing LayoutLMOnnxConfig and LayoutLMv3OnnxConfig support. Run the affected wmk perf evaluations and review the listed result.json files to confirm the failure. Done means all four LayoutLM/LayoutLMv3 models pass wmk perf and uv run pytest tests/ passes without hardcoded model names.