coder / coder/xum

Extremely slow (3 t/s) and can't setup llama.cpp + a couple of errors

Open
#3,186 0 comments 0 reactions 0 assignees View on GitHub
third-party
Dominant language
TypeScript
Stars
2k
Forks
134
Avg merge
14h 57m
Merged PRs (30d)
307

Description

Hey there, I have just tried Mux, and I face two problems that make the app non-usable at the moment.

1. Getting extremely low speeds with Qwen 3.6 35b a3b:
- Using OpenAI compatible URL for LM Studio, I get 3 t/s speed (note: my average LM Studio speed is 72 t/s)

2. Using Qwen 3.5 9b, it works fast but I get a couple of "Invalid type for 'input'." errors during chats:
- Speed: LM Studio ~ 86 t/s while Mux is ~80 t/s

Image

3. When I set llama.cpp up via OpenAI compatible URL, then try to chat, it keeps showing "Cannot determine type of 'item'
" error:

Image

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.