lmstudio-ai / lmstudio-ai/mlx-engine
MLX server (#353) as an Engine Protocol runtime: use case + offer to help
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 133
- Avg merge
- 21h 6m
- Merged PRs (30d)
- 1
Description
Hey, Dan here, co-founder of Certiv. We build a security layer that sits around whatever model you're using, local or remote. The point is to help large orgs actually adopt this stuff, especially local models. Right now they can put controls and safety classifiers on OpenAI and Anthropic, but the moment someone runs a model on their laptop none of that applies. We let them assert the same controls there.
LM Studio comes up constantly for us. A lot of the orgs we work with have people running it on their Macs and the security team has no idea what's going through it.
With Engine Protocol on, the llama.cpp path is great for us. llmster spins up llama-server on a local port, everything funnels through it (GUI chat, lms, API clients on 1234), and we can intercept that port and see the full request and response. I've got that working today on both Windows and Mac.
MLX is the gap. It's what most Mac users end up on by default, and it runs in-process so there's nothing on the wire for us to see. #353 looks like exactly the piece that closes it once it's wired in as a runtime the same way llama-server is.
Two questions:
1. Is MLX under Engine Protocol on the roadmap, and roughly when? Even a "yes, no date" helps me plan.
2. Is there anything in the way you'd take outside help on? The PR mentions the server doesn't emit semantic tool-call events yet, and llmster also hits /tokenize and /apply-template on llama-server that the MLX server doesn't have. Happy to pick up either if you point me at how you'd want it done.
Not asking for anything special here, just want the MLX path to look like the llama.cpp path on the wire. Thanks for open sourcing this, made it a lot easier to work out what was going on.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the issue's reference to #353 and the described llama-server integration path. Clarify whether MLX should be added as an Engine Protocol runtime and which gaps are in scope, including semantic tool-call events and the /tokenize and /apply-template endpoints. Done is not defined until maintainers choose a roadmap and implementation scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100