alibaba / alibaba/Sentinel

When spring-boot application context closed sentinel still has some not closed non-daemons thread

Open
#2,964 3 comments 0 reactions 0 assignees View on GitHub
good first issue kind/enhancement
Dominant language
Java
Stars
23.1k
Forks
8.1k
PR merge metrics
No merged PRs in 30d

Description

## Issue Description

Type: *bug report* or *feature request*

### Describe what happened (or what feature you want)

when I use `spring-cloud-starter-alibaba-sentine:2.2.9.RELEASE` with `sentinel-transport-netty-http` in my spring-boot application , the jvm can not exit when I call `actuator/shutdown` endpoint such as exec `curl -X POST http://ip:port/actuator/shutdown`.

This is because of `com.alibaba.csp.sentinel.transport.command.netty.HttpServer` thread has not closed when the application context closed.

### Describe what you expected to happen
Process finished with exit code 0

### How to reproduce it (as minimally and precisely as possible)

1. run you local sentinel dashboard on port 8080
2. clone a simple demo such as `git clone git@github.com:tobrainto/sentinel-demo.git`
3. Run it in Intelllij IDEA with args
```
-Dcsp.sentinel.dashboard.server=localhost:8080
-Dcsp.sentinel.api.port=8729
-Dproject.name=sentinel-demo
```
![image](https://user-images.githubusercontent.com/71546831/202891885-87390b64-702f-4177-b07c-c53affe45452.png)

4. exec `curl -X GET http://localhost:8090/hello` to trigger some init for sentinel
5. exec `curl -X POST http://localhost:8090/actuator/shutdown` to shutdown the application
6. you cannot find the info which mean the process finished as blow.
```
Disconnected from the target VM, address: '127.0.0.1:59120', transport: 'socket'

Process finished with exit code 0
```
and you can see some non-daemon thread still alive
![image](https://user-images.githubusercontent.com/71546831/202894150-c4b1fd91-2576-4f27-b016-c91d1d401d2e.png)

### Tell us your environment

Java11 、spring-cloud-starter-alibaba-sentine:2.2.9.RELEASEE 、spring-boot:2.6.6

### Try fixed
[I try to fixed it like this in spring-cloud-starter-alibaba-sentinel](https://github.com/alibaba/spring-cloud-alibaba/compare/2021.x...tobrainto:spring-cloud-alibaba:fix-sentinel-shutdown)

```
@PreDestroy
private void destroy() {
// Shutdown CommandCenter
CommandCenter commandCenter = CommandCenterProvider.getCommandCenter();
if (commandCenter != null) {
try {
commandCenter.stop();
}
catch (Exception e) {
log.warn("Sentinel CommandCenter shutdown error :", e);
}
}
}
```
it work well except for a exception trace print

```
java.lang.InterruptedException
at java.base/java.lang.Object.wait(Native Method)
at java.base/java.lang.Object.wait(Object.java:328)
at io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:253)
at io.netty.channel.DefaultChannelPromise.await(DefaultChannelPromise.java:131)
at io.netty.channel.DefaultChannelPromise.await(DefaultChannelPromise.java:30)
at io.netty.util.concurrent.DefaultPromise.sync(DefaultPromise.java:404)
at io.netty.channel.DefaultChannelPromise.sync(DefaultChannelPromise.java:119)
at io.netty.channel.DefaultChannelPromise.sync(DefaultChannelPromise.java:30)
at com.alibaba.csp.sentinel.transport.command.netty.HttpServer.start(HttpServer.java:86)
at com.alibaba.csp.sentinel.transport.command.NettyHttpCommandCenter$1.run(NettyHttpCommandCenter.java:50)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
```
so at the some I think `channel.closeFuture().sync();` may need a try catch to ingore the unnecessary exception trace print

![image](https://user-images.githubusercontent.com/71546831/202911890-11dfcd08-10e0-4dbc-bcef-e40375717e23.png)

### I also report it to spring-cloud-alibaba
[spring-cloud-alibaba/issues/2920
](https://github.com/alibaba/spring-cloud-alibaba/issues/2920
)

I first report it to spring-cloud-alibaba ,they reply me can report it here

thanks!

Contributor guide

Open the contributing guide

Research direction

Read com.alibaba.csp.sentinel.transport.command.netty.HttpServer.java and NettyHttpCommandCenter.java, then reproduce the issue with the linked sentinel-demo and the actuator shutdown request. Done means the Spring Boot process exits after shutdown, no non-daemon Sentinel thread remains, and the unnecessary InterruptedException trace is not printed.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring-boot
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.