spring-projects / spring-projects/spring-ai

Ollama inference thread not terminating after HTTP request is interrupted, causing GPU resource leak

Open
#2,944 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Bug description
After the main HTTP thread is interrupted, the background thread that sends requests to Ollama does not terminate properly. As a result, the GPU continues to be occupied with inference, even though the client has already disconnected or the HTTP context has been shut down.

Environment

Spring AI version: 0.7.0 (springai-7m)

Java version: java21

Vector store: None

Start a Spring Boot application with Spring AI integrated and set up to call Ollama.

Make a chat request via HTTP

Interrupt the HTTP request context or forcibly shut down the main thread.

Observe that the request to Ollama continues running in the background.

Check GPU status — the inference is still consuming GPU resources.

Expected behavior
The Ollama request thread should detect that the client has disconnected or that the context has been interrupted, and it should terminate immediately to release GPU resources.

Minimal Complete Reproducible example
A minimal Spring Boot project or test case that simulates an HTTP request to Ollama and interrupts the main thread mid-inference. GPU usage can be monitored via nvidia-smi or similar tools to confirm that inference continues after interruption.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the minimal Spring Boot reproduction described in the issue, interrupting an HTTP request during Ollama inference. Monitor GPU usage with nvidia-smi and trace the request and background-thread lifecycle; done means the Ollama inference stops and GPU resources are released when the client disconnects or the context is interrupted.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, ollama, spring-boot
Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.