spring-projects / spring-projects/spring-ai
Ollama inference thread not terminating after HTTP request is interrupted, causing GPU resource leak
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 9.5k
- Forks
- 2.9k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 6
Description
Bug description
After the main HTTP thread is interrupted, the background thread that sends requests to Ollama does not terminate properly. As a result, the GPU continues to be occupied with inference, even though the client has already disconnected or the HTTP context has been shut down.
Environment
Spring AI version: 0.7.0 (springai-7m)
Java version: java21
Vector store: None
Start a Spring Boot application with Spring AI integrated and set up to call Ollama.
Make a chat request via HTTP
Interrupt the HTTP request context or forcibly shut down the main thread.
Observe that the request to Ollama continues running in the background.
Check GPU status — the inference is still consuming GPU resources.
Expected behavior
The Ollama request thread should detect that the client has disconnected or that the context has been interrupted, and it should terminate immediately to release GPU resources.
Minimal Complete Reproducible example
A minimal Spring Boot project or test case that simulates an HTTP request to Ollama and interrupts the main thread mid-inference. GPU usage can be monitored via nvidia-smi or similar tools to confirm that inference continues after interruption.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the minimal Spring Boot reproduction described in the issue, interrupting an HTTP request during Ollama inference. Monitor GPU usage with nvidia-smi and trace the request and background-thread lifecycle; done means the Ollama inference stops and GPU resources are released when the client disconnects or the context is interrupted.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, ollama, spring-boot
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100