AlexsJones / AlexsJones/llmfit

[Feature]: Support LM Studio in Inference Bench (Quality Benchmarks / Routing)

未关闭
#928 3 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
enhancement
主要语言
Rust
星标
36.3k
派生
2.3k
平均合并
2 天 18 小时
30 天内合并 PR
85

描述

### Problem or motivation

`llmfit bench` currently only works with **Ollama, vLLM, and MLX**.

Even when LM Studio is correctly detected (`LM Studio: ✓ (N models)`) and a model is loaded + server is running, the Quality Benchmarks view shows:

> No Ollama models found. Install models first: ollama pull

### Current status

- Provider detection for LM Studio works (shows ✓ in the header)
- Quality / routing benchmarks (`I`) do **not** pick up LM Studio models

### Environment

- Hardware: Apple M4 Pro (48 GB)
- llmfit version: llmfit 1.1.10
- LM Studio: local server enabled, model loaded
- Model example: `qwen/qwen3.8-27b`

### Related

- #900 / #902 (LM Studio installed-model detection)

### Proposed solution

Inference Bench should be able to run quality benchmarks and produce the routing matrix against models served by **LM Studio** (OpenAI-compatible endpoint on port 1234 by default), the same way it already supports Ollama/vLLM/MLX.

### Alternatives considered

_No response_

### Feature area

Provider integration (Ollama, llama.cpp, MLX, Docker, LM Studio)

### Would you be willing to contribute this?

Yes, I'd like to submit a PR

### Additional context

_No response_

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。