spring-projects / spring-projects/spring-ai

500 Internal Server Error When Calling OpenAI-Compatible API via WebClient in Docker, But Not Locally

Open
#3,872 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Problem Description
When using Spring WebFlux’s WebClient to call my self-hosted OpenAI-compatible API endpoint (POST http://172.20.0.116:8000/v1/chat/completions), I consistently receive a 500 Internal Server Error after deploying the backend in a Docker container.
However, this issue does not occur in my local development environment (outside Docker).
My custom API strictly follows the OpenAI standard interface format and uses vllm for acceleration.
Error Log Excerpt
org.springframework.web.reactive.function.client.WebClientResponseException$InternalServerError: 500 Internal Server Error from POST http://172.20.0.116:8000/v1/chat/completions
at org.springframework.web.reactive.function.client.WebClientResponseException.create(WebClientResponseException.java:332)
...
Caused by: org.springframework.web.reactive.function.client.WebClientResponseException$InternalServerError: 500 Internal Server Error from POST http://172.20.0.116:8000/v1/chat/completions
at org.springframework.web.reactive.function.client.WebClientResponseException.create(WebClientResponseException.java:332)
...
Summary
Everything works fine in the local development environment; the 500 error only appears after deploying with Docker.
The logs show a server-side 500 error when WebClient calls /v1/chat/completions.
The backend API is custom but fully compatible with the OpenAI API spec, using vllm under the hood.
The service starts normally inside Docker, but every frontend request triggers this error.
Expectation
I would appreciate any help analyzing the possible causes of this issue, or suggestions for further troubleshooting. Thank you!
You can copy and paste the above directly into a GitHub issue.
If you need more troubleshooting tips or want to add more details, let me know!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the WebClient request to POST /v1/chat/completions and compare the Docker deployment with the local environment, including the endpoint and container networking. Inspect the server-side response and vllm logs to identify why the custom OpenAI-compatible service returns 500; done means the cause is reproduced and documented or the request succeeds in Docker.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, java, spring
Domain
api, backend, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.