AlexsJones / AlexsJones/llmfit
[Feature]: Support LM Studio in Inference Bench (Quality Benchmarks / Routing)
- 主要语言
- Rust
- 星标
- 36.5k
- 派生
- 2.3k
- 平均合并
- 3 天 3 小时
- 30 天内合并 PR
- 90
描述
### Problem or motivation
`llmfit bench` currently only works with **Ollama, vLLM, and MLX**.
Even when LM Studio is correctly detected (`LM Studio: ✓ (N models)`) and a model is loaded + server is running, the Quality Benchmarks view shows:
> No Ollama models found. Install models first: ollama pull
### Current status
- Provider detection for LM Studio works (shows ✓ in the header)
- Quality / routing benchmarks (`I`) do **not** pick up LM Studio models
### Environment
- Hardware: Apple M4 Pro (48 GB)
- llmfit version: llmfit 1.1.10
- LM Studio: local server enabled, model loaded
- Model example: `qwen/qwen3.8-27b`
### Related
- #900 / #902 (LM Studio installed-model detection)
### Proposed solution
Inference Bench should be able to run quality benchmarks and produce the routing matrix against models served by **LM Studio** (OpenAI-compatible endpoint on port 1234 by default), the same way it already supports Ollama/vLLM/MLX.
### Alternatives considered
_No response_
### Feature area
Provider integration (Ollama, llama.cpp, MLX, Docker, LM Studio)
### Would you be willing to contribute this?
Yes, I'd like to submit a PR
### Additional context
_No response_
贡献指南
调研方向
Start with the `llmfit bench` entry point and trace how the existing Ollama, vLLM, and MLX providers supply models to Quality Benchmarks and routing. Verify behavior with LM Studio’s OpenAI-compatible server on port 1234, including the loaded `qwen/qwen3.8-27b` model. Done means the Quality Benchmarks view no longer reports no Ollama models and produces the routing matrix for LM Studio models.
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- rust
- 领域
- ai
- Issue 类型
- 功能
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 活跃度
- 活跃
- 描述清晰度
- 基本清楚
- 新手友好度
- 65/100