spring-projects / spring-projects/spring-ai
500 Internal Server Error When Calling OpenAI-Compatible API via WebClient in Docker, But Not Locally
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 9.5k
- Forks
- 2.9k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 6
Description
Problem Description
When using Spring WebFlux’s WebClient to call my self-hosted OpenAI-compatible API endpoint (POST http://172.20.0.116:8000/v1/chat/completions), I consistently receive a 500 Internal Server Error after deploying the backend in a Docker container.
However, this issue does not occur in my local development environment (outside Docker).
My custom API strictly follows the OpenAI standard interface format and uses vllm for acceleration.
Error Log Excerpt
org.springframework.web.reactive.function.client.WebClientResponseException$InternalServerError: 500 Internal Server Error from POST http://172.20.0.116:8000/v1/chat/completions
at org.springframework.web.reactive.function.client.WebClientResponseException.create(WebClientResponseException.java:332)
...
Caused by: org.springframework.web.reactive.function.client.WebClientResponseException$InternalServerError: 500 Internal Server Error from POST http://172.20.0.116:8000/v1/chat/completions
at org.springframework.web.reactive.function.client.WebClientResponseException.create(WebClientResponseException.java:332)
...
Summary
Everything works fine in the local development environment; the 500 error only appears after deploying with Docker.
The logs show a server-side 500 error when WebClient calls /v1/chat/completions.
The backend API is custom but fully compatible with the OpenAI API spec, using vllm under the hood.
The service starts normally inside Docker, but every frontend request triggers this error.
Expectation
I would appreciate any help analyzing the possible causes of this issue, or suggestions for further troubleshooting. Thank you!
You can copy and paste the above directly into a GitHub issue.
If you need more troubleshooting tips or want to add more details, let me know!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the WebClient request to POST /v1/chat/completions and compare the Docker deployment with the local environment, including the endpoint and container networking. Inspect the server-side response and vllm logs to identify why the custom OpenAI-compatible service returns 500; done means the cause is reproduced and documented or the request succeeds in Docker.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, java, spring
- Domain
- api, backend, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100