Model Request: Gemma 4 26B-A4B-it (MoE, vision)

Model

google/gemma-4-26B-A4B-it — Apache 2.0

Rationale

FLM already supports GPT-OSS 20B (A3B), proving the MoE dispatch pipeline works on XDNA2. Gemma 4 26B-A4B is a natural next target — it's in the same weight class but adds vision:

The active parameter count (3.8B) is only ~5% larger than GPT-OSS (3.6B), so decode throughput should be comparable — potentially ~17+ t/s based on the GPT-OSS 18.2 t/s baseline (measured on Strix Halo XDNA2 NPU, FLM v0.9.32).

Why this model specifically

1. MoE infrastructure already exists — FLM's expert.xclbin + CPU router + three-tier expert dispatch (_npu_large_expert_task / _npu_short_expert_task / _cpu_expert_task) should generalize to Gemma 4's architecture with dimension adjustments.
2. Vision on NPU — Gemma 4 adds image understanding (variable-resolution, OCR, document parsing, charts). FLM already supports Qwen3.5 VL 4B, so the vision pipeline has precedent. A 26B-class vision model on NPU would be a significant capability.
3. Strongest open MoE at this size — It significantly outperforms Gemma 3 27B dense on every benchmark.
4. Thinking mode — Built-in reasoning/chain-of-thought via enable_thinking=True, useful for coding and agentic workflows.
5. Apache 2.0 — No license friction.

Architecture details

• Hybrid attention: interleaved sliding window (1024 tokens) + full global attention
• Sliding window on local layers, global layers use unified KV + Proportional RoPE
• Expert FFN dimensions would need new xclbin tile configs (hidden_size=3072, intermediate unknown but derivable from config)
• 262K vocabulary

Key difference from GPT-OSS

The main new challenge is the higher expert count (128 total vs 32) and the vision encoder (~550M params). The expert routing still selects a fixed top-k per token, so the dispatch pattern is the same — just more expert weight blocks to index into. The vision encoder would need its own processing path (similar to how Qwen3.5-VL is handled).

Start by reading the existing expert.xclbin path and the three dispatch entry points (_npu_large_expert_task, _npu_short_expert_task, and _cpu_expert_task), then compare the existing GPT-OSS and Qwen3.5-VL support. Done means Gemma 4 text and vision requests run on XDNA2 with appropriate expert configuration and validated performance.