apache / apache/dubbo

[resubmit] fail to gracefully shutdown java dubbo app

Open
#11,951 4 comments 0 reactions 0 assignees View on GitHub
type/bug
Dominant language
Java
Stars
41.6k
Forks
26.4k
Avg merge
15h 13m
Merged PRs (30d)
4

Description

dubbo version: 3.0.9
error: 2023-03-28 17:47:33.403 ERROR 4212 --- [o-8082-exec-303] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.apache.dubbo.rpc.RpcException: Failed to invoke the method sayHello in the service org.example.api.DemoService. Tried 1 times of the providers [192.168.2.205:20880] (1/1) from the registry mse-4aac8ab0-zk.mse.aliyuncs.com:2181 on the consumer 192.168.2.206 using the dubbo version 3.0.9. Last error is: Failed to invoke remote method: sayHello, provider: dubbo://192.168.2.205:20880/org.example.api.DemoService anyhost=true&application=provider&background=false&category=providers,configurators,routers&check=false&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&interface=org.example.api.DemoService&methods=sayHello&pid=4212&qos.enable=false&release=3.0.9&retries=0&service-name-mapping=true&side=provider&sticky=false&timeout=30000000, cause: org.apache.dubbo.remoting.RemotingException: Fail to decode request due to: java.io.IOException: Service org.example.api.DemoService with version 0.0.0 not found, invocation rejected.

dubbo.properties:
dubbo.service.shutdown.wait=600000
server.shutdown=graceful

background:

to shutdown jar app gracefully , initially ,run curl localhost:22222/offline && sleep 3 , and kill PID, meanwhile, the new requests arrive continuously, then the above errors happen, which cause some requests fail.

any way to aovid the error? thanks lot
.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reported sequence with Dubbo 3.0.9: configure dubbo.service.shutdown.wait=600000 and server.shutdown=graceful, call curl localhost:22222/offline, wait three seconds, and terminate the process while requests continue. Trace the graceful-shutdown and RPC provider behavior, then verify that in-flight and newly arriving requests do not fail with the reported service-not-found error.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.