huggingface / huggingface/optimum-intel

Add OpenVINO export support for spark2_5 architecture (Spark-X2.5-4B / 1.7B)

Open
#2,003 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
620
Forks
270
Avg merge
3d 10h
Merged PRs (30d)
22

Description

### Feature request

Please add OpenVINO export support for the `spark2_5` architecture (`Spark2_5ForCausalLM`), used by the Spark-X2.5 series.

- Model card: https://huggingface.co/XHToken/Spark-X2.5-4B (also a 1.7B variant)
- `model_type: spark2_5`, `architectures: ["Spark2_5ForCausalLM"]`

### Motivation

Spark-X2.5 is an on-device agentic model family with strong published benchmarks. We are evaluating it as a local replacement for our production model on **Intel Core Ultra (Meteor Lake) iGPU**, where **OpenVINO GenAI is our production inference stack**. Today there is no OpenVINO path for this architecture — neither `optimum-intel`, `openvino.genai`, nor `openvino` contains any reference to `spark2_5`, and it is not in the [OpenVINO GenAI supported-models list](https://openvinotoolkit.github.io/openvino.genai/docs/supported-models/).

### Architecture summary (from `config.json`)

Hybrid interleaved attention:

- **36 layers**, pattern = 3× sliding-window + 1× full attention, repeated ⇒ 27 sliding / 9 full
- `sliding_window: 512`
- GQA: `num_attention_heads: 16`, `num_key_value_heads: 4`, `head_dim: 256`
- `hidden_size: 2560`, `intermediate_size: 10240`
- `vocab_size: 131072`, `max_position_embeddings: 1048576`
- **Dual RoPE base** (NeoX-style): `10000` for sliding layers / `5000000` for full-attention layers
- **Gated attention**: `headwise_attn_output_gate: true`, `gate_attn_act_mode: sigmoid`
- `rms_norm_eps: 1e-06`

### Why explicit support is needed (not "similar architecture may also work")

The custom ops — the **headwise gated attention output gate** and the **interleaved SWA with dual RoPE base** — are exactly the parts that silently break on backends that have not implemented them. In our testing with llama.cpp (which added native `spark2_5` support in Sept 2026):

- CPU and the **OpenCL** GPU backend produce **correct** output.
- The **Vulkan** backend loads but produces **incoherent garbage** (numerically wrong attention) — both with and without flash-attention.
- The **Level-Zero (SYCL)** backend **segfaults at model load**.

So this architecture will not "just work" by architecture similarity — it needs explicit enablement.

### Ecosystem status

- **llama.cpp**: native `spark2_5` support added Sept 2026 (GGUF).
- **Ollama**: open request (ollama/ollama#18195).
- **OpenVINO / optimum-intel**: no code, no issue, not in the supported-models list.

### Ask

Is enabling `spark2_5` for OpenVINO export on the roadmap? We are happy to help validate on Intel Core Ultra (Meteor Lake) iGPU with OpenVINO GenAI once a conversion path exists.

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue names no repository files, tests, or entry points; start by locating existing OpenVINO export implementations for comparable causal-language-model architectures and their conversion tests. Define the integration points required for `spark2_5`, then validate export and generated-model inference on the stated Intel Core Ultra/OpenVINO GenAI setup.

Written by the indexing model from the issue text.

Assessment

Domain
machine-learning, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.