P1-FEATURE-GGUF: GGUF Format — Load & Convert to ONNX

Summary

Implement GGUF model format loading and conversion to ONNX, supporting the top 10 popular GGUF architectures as an experimental best-effort feature.

Context

GGUF (GPT-Generated Unified Format) is the standard format for quantized LLMs deployed via llama.cpp. Supporting GGUF → ONNX conversion lets ModelKit work with the large ecosystem of GGUF-quantized models without requiring original PyTorch weights.

This is experimental (best-effort) for May 1. From:

• plans/release/0315_release_plan/P1_CHECKLIST.md (P1-FEATURE-003)
• plans/release/0501_release_plan/P0_CHECKLIST.md (P1-FEATURE-006)

Current State

• No GGUF support in ModelKit — only HuggingFace + local ONNX loading
• GGUF format spec: https://github.com/ggerganov/ggml/blob/master/docs/gguf.md

Desired State

• wmk build <path_to_model.gguf> loads and converts GGUF model to ONNX
• INT4/INT8 quantization metadata preserved in ONNX output
• Top 10 popular GGUF architectures supported (best-effort)
• Converted ONNX model runs through the standard pipeline (analyze → optimize → perf)

Acceptance Criteria

• GGUF file parser implemented (header + tensor extraction)
• GGUF → ONNX conversion for at least 5 architectures (Llama, Mistral, Phi, Gemma, Qwen)
• INT4 quantization metadata preserved in ONNX output
• Converted model passes static analysis and runs on at least one EP (CPU)
• wmk build --format gguf path/to/model.gguf works E2E
• Experimental flag clearly communicated in CLI output
• All existing tests pass (GGUF is additive, no regressions)

Technical Notes

• GGUF tensor types: F32, F16, Q4_0, Q4_1, Q8_0, Q5_0, Q5_1, Q4_K, Q6_K, etc.
• Consider using gguf Python library for parsing
• INT4 representation in ONNX: use MatMulNBits op (ORT extension) or pack as INT8 with scale
• Experimental status: gate behind --experimental flag or warning message

Related Files

• plans/release/0315_release_plan/feature-scale.md — P1.2 GGUF Support
• plans/release/0501_release_plan/feature-scale.md — P1.6 GGUF Format
• plans/release/0315_release_plan/P1_CHECKLIST.md — P1-FEATURE-003
• plans/release/0501_release_plan/P0_CHECKLIST.md — P1-FEATURE-006