ROCm / ROCm/FastFlowLM

Support request: Bielik-11B-v3.0-Instruct-heretic-MPOA on Ryzen AI NPU

Open
#704 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
1.9k
Forks
152
Avg merge
4h 14m
Merged PRs (30d)
11

Description

Hello FastFlowLM team,

I would like to request NPU support for the Polish Llama-based model Bielik-11B-v3.0-Instruct-heretic-MPOA.

Model sources:

I converted the Q6_K GGUF to Q4NX_FLL successfully. The resulting model has 50 layers, hidden size 4096, intermediate size 14336, 32 attention heads, and 8 key/value heads.

System:

  • Linux
  • FastFlowLM 1.0.4
  • AMD RyzenAI-npu6 / XDNA2
  • XRT detects the NPU and flm validate --json reports ready: true.

I tested the existing Llama-3.1-8B-NPU2 xclbins as a diagnostic only. The model loads and acquires the NPU, but generation fails with:

runlist failed execution (ERT_CMD_STATE_TIMEOUT)
Kernel Instance: MLIR_AIE

Could you please advise:

  1. Is there an existing compatible xclbin set for this Bielik/Llama architecture?
  2. If not, could support be added, including the required attn.xclbin, dequant.xclbin, layer.xclbin, and mm.xclbin?
  3. Is there a supported public workflow for compiling FLM-compatible xclbins for a custom Q4NX_FLL model?

I can provide the Q4NX tensor metadata, model configuration, and reproducible logs if helpful.

Thank you.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the flm validate --json result, the Q4NX_FLL tensor metadata and model configuration, and the timeout logs from the diagnostic Llama-3.1-8B-NPU2 xclbins. Done would be identifying a compatible xclbin set or establishing whether a supported workflow exists for compiling the requested attn.xclbin, dequant.xclbin, layer.xclbin, and mm.xclbin files.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux
Domain
ai-infra-agents, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.