AlexsJones / AlexsJones/llmfit

[Feature]: Support LM Studio in Inference Bench (Quality Benchmarks / Routing)

未关闭
#928 3 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
enhancement
主要语言
Rust
星标
36.5k
派生
2.3k
平均合并
3 天 3 小时
30 天内合并 PR
90

描述

### Problem or motivation

`llmfit bench` currently only works with **Ollama, vLLM, and MLX**.

Even when LM Studio is correctly detected (`LM Studio: ✓ (N models)`) and a model is loaded + server is running, the Quality Benchmarks view shows:

> No Ollama models found. Install models first: ollama pull

### Current status

- Provider detection for LM Studio works (shows ✓ in the header)
- Quality / routing benchmarks (`I`) do **not** pick up LM Studio models

### Environment

- Hardware: Apple M4 Pro (48 GB)
- llmfit version: llmfit 1.1.10
- LM Studio: local server enabled, model loaded
- Model example: `qwen/qwen3.8-27b`

### Related

- #900 / #902 (LM Studio installed-model detection)

### Proposed solution

Inference Bench should be able to run quality benchmarks and produce the routing matrix against models served by **LM Studio** (OpenAI-compatible endpoint on port 1234 by default), the same way it already supports Ollama/vLLM/MLX.

### Alternatives considered

_No response_

### Feature area

Provider integration (Ollama, llama.cpp, MLX, Docker, LM Studio)

### Would you be willing to contribute this?

Yes, I'd like to submit a PR

### Additional context

_No response_

贡献指南

打开贡献指南

调研方向

Start with the `llmfit bench` entry point and trace how the existing Ollama, vLLM, and MLX providers supply models to Quality Benchmarks and routing. Verify behavior with LM Studio’s OpenAI-compatible server on port 1234, including the loaded `qwen/qwen3.8-27b` model. Done means the Quality Benchmarks view no longer reports no Ollama models and produces the routing matrix for LM Studio models.

由索引模型根据 Issue 内容生成。

评估

技术栈
rust
领域
ai
Issue 类型
功能
难度
3/5
预计耗时
1-2 天
活跃度
活跃
描述清晰度
基本清楚
新手友好度
65/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。