ROCm / ROCm/FastFlowLM

FLM 1.0.2 on Linux: Qwen3.5 4B and 9B hang during prefill, while Llama 3.2 and Qwen3.6 work

Open
#678 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
1.9k
Forks
152
Avg merge
4h 14m
Merged PRs (30d)
11

Description

Environment:

  • Fedora 44
  • Kernel: 7.1.8-200.fc44.x86_64
  • Hardware: AMD Ryzen AI 7 PRO 350, RyzenAI-npu6 (6x8)
  • FLM: 1.0.2
  • XRT: 2.26.0
  • amdxdna: 0.8
  • NPU firmware: 1.1.2.64

Validation:
flm validate succeeds:

  • /dev/accel/accel0 with 8 columns
  • NPU firmware 1.1.2.64
  • amdxdna 0.8
  • memlock infinity

Repro:
flm run qwen3.5:4b
or
flm run qwen3.5:9b

Prompt:
hi

Actual result:
Both models load successfully, then print:
[FLM] Prefill chunk 1/1 with 13 tokens
and hang indefinitely. They never reach:
[FLM] Creating checkpoint at context length 13

The Qwen3.5 model directories were removed and downloaded again after FLM reported old Qwen models as incompatible.

Working controls on the exact same system:

  • llama3.2:3b processes a 41-token prefill and replies correctly.
  • qwen3.6-moe:35b-a3b passes prefill, creates a checkpoint, and replies correctly.

RAM is not exhausted:

  • Qwen3.5 4B/9B uses roughly 45% RAM when hung.
  • qwen3.6-moe:35b-a3b works despite RAM usage above 90%.

This appears specific to the Qwen3.5 prefill/runtime path in FLM 1.0.2 on Linux.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing flm run qwen3.5:4b and flm run qwen3.5:9b with the listed Fedora, XRT, amdxdna, and firmware versions, then compare their prefill behavior with the working Llama 3.2 and Qwen3.6 controls. Trace the Qwen3.5 prefill/runtime path around the last logged step; done means both models complete prefill, create the checkpoint, and answer the prompt.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, linux
Domain
machine-learning, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.