ROCm / ROCm/FastFlowLM

[Issue]: v1.0.4 causes NPU hardware crash (0xc00002b6) and Kernel Deadlock on AMD Ryzen AI (v1.0.3 works fine)

Open
#708 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
1.9k
Forks
152
Avg merge
4h 14m
Merged PRs (30d)
11

Description

Problem Description

Running any model (even 0.8b) on FastFlowLM v1.0.4 causes a severe NPU hardware disconnect and kernel deadlock. The XRT driver reports 0xc00002b6 (STATUS_DEVICE_REMOVED), indicating the NPU crashed or was forcefully kicked offline by the Windows kernel.
Because the NPU drops offline while memory mapping (mmap) is active, it completely hangs the NTFS file system, causing runlist error and a system-wide deadlock. The flm.exe process becomes an unkillable zombie process (cannot be terminated via taskkill /F), requiring a hard physical reboot.

To Reproduce

System Environment:

OS: Windows 11

Hardware: MECHREVO XINGYAO 15 (AMD Ryzen AI 9 365)

BIOS Version: Updated to the latest official version

AMD IPU Driver: Updated to the latest official Ryzen AI driver

FLM Version: v1.0.4 (Bugged) / v1.0.3 (Works perfectly)

Additional Context (Important Workaround)
I have updated my motherboard BIOS and AMD NPU drivers to the latest versions.
After completely uninstalling v1.0.4, clearing the .flm\models cache, and downgrading to v1.0.3, the exact same model (qwen3.5:9b) runs perfectly stable on the NPU without any crashes or garbage output. This confirms the issue is strictly isolated to the XRT/NPU invocation updates introduced in v1.0.4.

Operating System

Windows 11

CPU

AMD Ryzen AI 9 365

GPU

AMD Radeon 880M

ROCm Version

N/A (Windows 11 XRT driver, NPU-only path)

ROCm Component

No response

Steps to Reproduce

Environment: AMD Ryzen AI NPU (Tested on MECHREVO XINGYAO Series, Ryzen AI 9 365).

Install FLM v1.0.4.

Run command: flm run qwen3.5:0.8b (Even with --pmode balanced -c 4096).

Type hello.

The terminal throws XRT errors, Ctrl+C fails, and the system deadlocks.
[FLM] Prefill chunk 1/1 with 9 tokens
XRT build version: 2.19.0
Build hash: 77c7088d804602a53c3eb489b9cb37b709bcd751
Build date: 2025-12-04 16:52:22
[XRT] ERROR: Failed to submit command to hw queue (0xc00002b6): 豸ѱɾ
[XRT] ERROR: Failed to submit command to hw queue (0xc00002b6): 豸ѱɾ
[FLM] Failed to submit command to hw queue (0xc00002b6): 豸ѱɾ
[ERROR] Insertion error: Failed to submit command to hw queue (0xc00002b6): 豸ѱɾ
(Note: 豸ѱɾ is GBK encoding for "设备已被删除" / "Device has been removed").

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support
rocminfo --support output
Paste output here
Additional Information

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the flm run entry point and compare the XRT/NPU invocation changes between v1.0.4 and the working v1.0.3, using the reported Ryzen AI 9 365 environment. Reproduce with flm run qwen3.5:0.8b and confirm that submission no longer reports 0xc00002b6 or causes the kernel deadlock.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
ai-infra-agents
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.