spring-projects / spring-projects/spring-ai
Unable to use spring ai with LMStudio using spring-ai openai module
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 9.5k
- Forks
- 2.9k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 6
Description
Bug description
When trying to connect to LM studio on http://localhost:1234 for a chat completion request the program hangs, if you exit from LMStudio Spring AI complains that to bytes are received, which makes me thinks that it is waiting for a response. Please note that using the official OpenAI endpoint works correctly
I tried to send a chat completion request using curl and it works after loading the model with lms load
curl http://localhost:1234/v1/chat/completions ^
More? -H "Content-Type: application/json" ^
More? -H "Authorization: Bearer YOUR_OPENAI_API_KEY" ^
More? -d "{ "model": "llama-3.2-1b-instruct", "messages": [ { "role": "user", "content": "hello" } ], "temperature": 0.7 }"
{
"id": "chatcmpl-qjmrt2dzv5plaf7w22f8p8",
"object": "chat.completion",
"created": 1741679687,
"model": "llama-3.2-1b-instruct",
"choices": [
{
"index": 0,
"logprobs": null,
"finish_reason": "stop",
"message": {
"role": "assistant",
"content": "Hello! How can I assist you today?"
}
}
],
"usage": {
"prompt_tokens": 11,
"completion_tokens": 9,
"total_tokens": 20
},
"system_fingerprint": "llama-3.2-1b-instruct"
}
Environment
I am using Windows 11 24H2, Java 23.0 (amazon corretto), spring AI M6 (but M4 shows the same behaviour) and LM Studio 0.3.8 (but 0.3.12 shows the same issue) I tried with both CORS enabled or disabled, and both serving the LMStudio on the local network and not serving it on the local network, it made no difference.
Steps to reproduce
in the attached zip file there's a minimal spring application built with spring initializr that incorporates Spring AI M6 with the OpenAI module
Just run SpringBootApplication.java
Expected behavior
Main.java is a CommandLineRunner where I build a ChatClient and send the prompt hello to the model, I expect the program to exit once the generation is complete but the program hangs forever.
Minimal Complete Reproducible example
See the attached zip file
demo.zip
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the attached demo.zip, especially Main.java and SpringBootApplication.java, and run the minimal application against LMStudio at http://localhost:1234. Compare its behavior with the working curl chat-completion request and the official OpenAI endpoint. Done means the CommandLineRunner completes the hello request and the program exits normally.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spring-boot
- Domain
- ai, api, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100