t5 / summarization + translation: all models pass wmk perf

Summary

All T5 models fail on summarization and translation tasks because the new EncoderDecoderCache type returned by recent versions of transformers is not serializable by TorchScript (used during ONNX export). This blocks the entire T5 family.

Eval Results (2026-03-11)

8/8 models fail — 0 pass.

Error: Only tuples, lists and Variables are supported as JIT inputs/outputs. Dictionaries and strings are also accepted, but their usage is not recommended. Here, received an input of unsupported type: EncoderDecoderCache

Root Cause

Recent transformers versions introduced EncoderDecoderCache as the KV-cache type for encoder-decoder models. The ONNX export path uses TorchScript tracing, which cannot handle this custom cache type. The fix requires either:

• Disabling/bypassing cache during export (use_cache=False)
• Using past_key_values=None and static export

Current State

• modelkit/export/ — ONNX export logic
• modelkit/models/hf/ — no t5.py model config exists (no explicit T5 ONNX config registration)
• T5 relies on Optimum's default export path, which may not handle EncoderDecoderCache

Desired State

All 8 T5 models above pass wmk perf for summarization and translation tasks.

Acceptance Criteria

• All 8 T5 models listed above pass wmk perf
• No hardcoded logic for T5 specifically — fix must work for any encoder-decoder model hitting the same issue (CLAUDE.md Cardinal Rule #1)
• uv run pytest tests/ passes (CLAUDE.md Cardinal Rule #3)
• Eval re-run confirms 8/8 pass

Technical Notes

• Root fix: set use_cache=False during export for encoder-decoder models, or register a T5-specific ONNX config that handles cache correctly
• Consider adding modelkit/models/hf/t5.py similar to bert.py / clip.py with a custom T5IOConfig registered via @register_onnx_overwrite
• Optimum's T5OnnxConfig may already have a workaround — investigate before writing custom logic
• Same root cause affects bart, marian, mbart — see related issues

Related Files

• modelkit/models/hf/bert.py — pattern for adding model-specific ONNX config
• modelkit/export/io.py — register_onnx_overwrite() decorator
• modelkit/models/hf/__init__.py:27-53 — where new t5.py import would be added
• eval_results/2026-03-11/models/google-t5__t5-small__summarization/result.json — failure evidence

References

• CLAUDE.md Cardinal Rule #1: No hardcoded model type strings
• CLAUDE.md Cardinal Rule #3: Run uv run pytest tests/ after implementation

Start by reading modelkit/export/io.py and the related model-specific pattern in modelkit/models/hf/bert.py, then inspect modelkit/models/hf/init.py:27-53 and the recorded failure result.json. Run uv run pytest tests/ while tracing the ONNX export path for encoder-decoder models. Done means the eight listed T5 evaluations pass without T5-specific hardcoded logic and the full test suite passes.