poor TPS
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 152
- Avg merge
- 4h 14m
- Merged PRs (30d)
- 11
Description
Environment:
xrt-smi examine
System Configuration
OS Name : Linux
Release : 7.0.11-200.fc44.x86_64
Machine : x86_64
CPU Cores : 24
Memory : 32 GB
Distribution : Fedora Linux 44 (Workstation Edition)
GLIBC : 2.43
Model : SER9
BIOS Vendor : American Megatrends International, LLC.
BIOS Version : STX.3xx.SER9.V218.P8C0M0C15.74.Link.SCPC
Processor : AMD Ryzen AI 9 HX 370 w/ Radeon 890M
XRT
Version : 2.25.0
Branch : HEAD
Hash : c63dac03a1dd0d4782befd127f65362ea12478c4
Hash Date : Thu, 28 May 2026 08:54:04 -0700
amdxdna Version : 2.25.0_20260607, 302b82f9120d6706c4e235c4283eb0fe7452b966
virtio-pci Version : 7.0.11-200.fc44.x86_64
NPU Firmware Version : 1.1.2.64
Device(s) Present
| BDF | Name | Architecture | Topology |
|---|---|---|---|
| [0000:66:00.1] | NPU Strix | aie2p | 6x8 |
flm validate
[Linux] Kernel: 7.0.11-200.fc44.x86_64
[Linux] NPU: /dev/accel/accel0 with 8 columns
[Linux] NPU FW Version: 1.1.2.64
[Linux] amdxdna version: 0.15
[Linux] Memlock Limit: infinity
I'm launching benchmark
flm bench qwen3.5:2b
my results:
| Model | HW | 1k | 2k | 4k | 8k | 16k | 32k |
|---|---|---|---|---|---|---|---|
| Qwen3.5-2B | NPU (FLM) | 8.63 | 8.38 | 8.21 | 7.83 | 7.20 | 6.20 |
your results:
| Model | HW | 1k | 2k | 4k | 8k | 16k | 32k |
|---|---|---|---|---|---|---|---|
| Qwen3.5-2B | NPU (FLM) | 26.8 | 26.2 | 25.4 | 23.7 | 21.3 | 17.0 |
Why so poor TPS (Tokens Per Second) for me.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing flm bench qwen3.5:2b under the reported Fedora, XRT, amdxdna, firmware, and NPU configuration, then compare the measured TPS with the listed results. No source file or test is named; done means identifying the configuration or performance cause and documenting a verified explanation or fix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, linux
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 42/100