spring-projects / spring-projects/spring-ai

Unable to use spring ai with LMStudio using spring-ai openai module

Open
#2,441 12 comments 5 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Bug description
When trying to connect to LM studio on http://localhost:1234 for a chat completion request the program hangs, if you exit from LMStudio Spring AI complains that to bytes are received, which makes me thinks that it is waiting for a response. Please note that using the official OpenAI endpoint works correctly

I tried to send a chat completion request using curl and it works after loading the model with lms load

curl http://localhost:1234/v1/chat/completions ^
More? -H "Content-Type: application/json" ^
More? -H "Authorization: Bearer YOUR_OPENAI_API_KEY" ^
More? -d "{ "model": "llama-3.2-1b-instruct", "messages": [ { "role": "user", "content": "hello" } ], "temperature": 0.7 }"
{
"id": "chatcmpl-qjmrt2dzv5plaf7w22f8p8",
"object": "chat.completion",
"created": 1741679687,
"model": "llama-3.2-1b-instruct",
"choices": [
{
"index": 0,
"logprobs": null,
"finish_reason": "stop",
"message": {
"role": "assistant",
"content": "Hello! How can I assist you today?"
}
}
],
"usage": {
"prompt_tokens": 11,
"completion_tokens": 9,
"total_tokens": 20
},
"system_fingerprint": "llama-3.2-1b-instruct"
}

Environment
I am using Windows 11 24H2, Java 23.0 (amazon corretto), spring AI M6 (but M4 shows the same behaviour) and LM Studio 0.3.8 (but 0.3.12 shows the same issue) I tried with both CORS enabled or disabled, and both serving the LMStudio on the local network and not serving it on the local network, it made no difference.

Steps to reproduce
in the attached zip file there's a minimal spring application built with spring initializr that incorporates Spring AI M6 with the OpenAI module
Just run SpringBootApplication.java
Expected behavior
Main.java is a CommandLineRunner where I build a ChatClient and send the prompt hello to the model, I expect the program to exit once the generation is complete but the program hangs forever.

Minimal Complete Reproducible example
See the attached zip file
demo.zip

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the attached demo.zip, especially Main.java and SpringBootApplication.java, and run the minimal application against LMStudio at http://localhost:1234. Compare its behavior with the working curl chat-completion request and the official OpenAI endpoint. Done means the CommandLineRunner completes the hello request and the program exits normally.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring-boot
Domain
ai, api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.