Add FIM models
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 152
- Avg merge
- 4h 14m
- Merged PRs (30d)
- 11
Description
Hi,
Fill In the Middle (FIM) models used for code auto completion are a perfect fit for NPUs as they are small, fast, and have to infer all the time when coding.
They are so much needed that Llama.cpp CLI has the following options to directly install them:
--fim-qwen-1.5b-default
--fim-qwen-3b-default
--fim-qwen-7b-default
--fim-qwen-7b-spec
--fim-qwen-14b-spec
--fim-qwen-30b-default
Recently, Sweep released two state of the art FIM tiny models that seem to be even better than the Qwen ones:
I use Sweep Next-Edit 1.5B every day and I can confirm it is super fast and excellent for coding. You can see how well Sweep FIM models perform on https://blog.sweep.dev/posts/oss-next-edit
On my Strix Halo I see no reason to use my NPU instead of my iGPU, except for FIM. Adding some FIM models would finally make my NPU somewhat super useful and would really make my day and the day of many developers.
Thank you for your great project!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no FastFlowLM files, tests, or entry points. Start by comparing existing model integrations with the referenced Llama.cpp FIM options and the two Sweep model pages. Done means the selected FIM models are available through FastFlowLM and work on AMD Ryzen AI NPUs.
Written by the indexing model from the issue text.
Assessment
- Domain
- ai, embedded-iot, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 42/100