actix / actix/actix-web

Server does not clean up resources after calling ServerHandle::stop in certain cases

Aperta
#2,759 5 commenti 3 reazioni 0 assegnatari Vedi su GitHub
needs-investigation
Lingua principale
Rust
Stelle
24.8k
Fork
1.9k
Merge medio
23h 10m
PR unite (30g)
26

Descrizione

If a service takes long time to complete (longer than value specified by`HttpServer::shutdown_timeout` to be precise), it seems the server does not properly clean up after itself. In particular, I have an issue with this when the server holds on to a tokio channel sender and I'm waiting for the channel's receiver to get "channel closed" message in a separate tokio task after stopping the server.

## Expected Behavior
After calling `ServerHandle::stop` and waiting until the end of the `shutdown_timeout` period, all resources held by the server should be dropped.

## Current Behavior
Server seems to still hold on to resources.

## Possible Solution
None known...

## Steps to Reproduce (for bugs)
Given the following example application:

```rust
use actix_web::{Responder, HttpResponse, web::{Data}};
use tokio::sync::{mpsc, broadcast};
use log::*;

#[actix_web::post("/sleep10")]
pub async fn sleep10(tx: Data>) -> impl Responder {
info!("sleeping 10 seconds");
tokio::time::sleep(std::time::Duration::from_secs(10)).await;
info!("slept 10 seconds");
tx.send(()).unwrap();
HttpResponse::Ok()
}

#[actix_web::post("/sleep30")]
pub async fn sleep30(tx: Data>) -> impl Responder {
info!("sleeping 30 seconds");
tokio::time::sleep(std::time::Duration::from_secs(30)).await;
info!("slept 30 seconds");
tx.send(()).unwrap();
HttpResponse::Ok()
}

#[tokio::main]
async fn main() -> anyhow::Result<()> {
env_logger::init();

let (stop_tx, mut stop_rx) = broadcast::channel(1);
let (tx, mut rx) = mpsc::unbounded_channel();

// My spawned task that consumes the channel’s receiver.
let consumer_handle = tokio::spawn(async move {
loop {
tokio::select! {
Some(()) = rx.recv() => info!("task received item"),
Ok(()) = stop_rx.recv() => {
info!("task draining items");
while let Some(()) = rx.recv().await { // This will block as sender is never dropped.
info!("task drained item");
}
info!("task finished draining");
break;
}
}
}
});

let tx = Data::new(tx);
let server = actix_web::HttpServer::new(move || {
let tx = tx.clone();
actix_web::App::new()
.service(sleep10)
.service(sleep30)
.app_data(tx) // This sender is never dropped.
})
.bind("localhost:20000")?
.workers(4)
.disable_signals()
.shutdown_timeout(20) // Modified shutdown timeout, less than 30 seconds.
.run();

let server_handle = server.handle();
let signal = tokio::signal::ctrl_c();

tokio::pin!(server);
tokio::select! {
r = signal => {
info!("received interrupt signal");
r.unwrap();
let ((), r) = tokio::join!(server_handle.stop(true), server);
r.unwrap();
info!("stopping task");
stop_tx.send(()).unwrap();
consumer_handle.await.unwrap();
}
r = &mut server => {
info!("server finished");
r.unwrap();
stop_tx.send(()).unwrap();
consumer_handle.await.unwrap();
}
}

Ok(())
}
```

When I call `curl -X POST localhost:20000/sleep10` and send `SIGINT` to the application, it cleanly shutsdown (OK).

```bash
$ RUST_LOG=debug cargo run
[2022-05-13T08:18:56Z INFO actix_server::builder] Starting 4 workers
[2022-05-13T08:18:56Z INFO actix_server::server] Tokio runtime found; starting in existing Tokio runtime
[2022-05-13T08:19:00Z INFO shutdown] sleeping 10 seconds
^C[2022-05-13T08:19:01Z INFO shutdown] received interrupt signal
[2022-05-13T08:19:01Z INFO actix_server::worker] Graceful worker shutdown; finishing 1 connections
[2022-05-13T08:19:01Z DEBUG actix_server::accept] Paused accepting connections on [::1]:20000
[2022-05-13T08:19:01Z DEBUG actix_server::accept] Paused accepting connections on 127.0.0.1:20000
[2022-05-13T08:19:01Z INFO actix_server::accept] Accept thread stopped
[2022-05-13T08:19:01Z INFO actix_server::worker] Shutting down idle worker
[2022-05-13T08:19:01Z INFO actix_server::worker] Shutting down idle worker
[2022-05-13T08:19:01Z INFO actix_server::worker] Shutting down idle worker
[2022-05-13T08:19:10Z INFO shutdown] slept 10 seconds
[2022-05-13T08:19:10Z INFO shutdown] task received item
[2022-05-13T08:19:11Z INFO shutdown] stopping task
[2022-05-13T08:19:11Z INFO shutdown] task draining items
[2022-05-13T08:19:11Z INFO shutdown] task finished draining
$
```

If I call `curl -X POST localhost:20000/sleep30` and send `SIGINT`, the application never terminates (BUG).

```bash
$ RUST_LOG=debug cargo run
[2022-05-13T08:21:14Z INFO actix_server::builder] Starting 4 workers
[2022-05-13T08:21:14Z INFO actix_server::server] Tokio runtime found; starting in existing Tokio runtime
[2022-05-13T08:21:17Z INFO shutdown] sleeping 30 seconds
^C[2022-05-13T08:21:19Z INFO shutdown] received interrupt signal
[2022-05-13T08:21:19Z DEBUG actix_server::accept] Paused accepting connections on [::1]:20000
[2022-05-13T08:21:19Z DEBUG actix_server::accept] Paused accepting connections on 127.0.0.1:20000
[2022-05-13T08:21:19Z INFO actix_server::worker] Shutting down idle worker
[2022-05-13T08:21:19Z INFO actix_server::worker] Shutting down idle worker
[2022-05-13T08:21:19Z INFO actix_server::worker] Shutting down idle worker
[2022-05-13T08:21:19Z INFO actix_server::worker] Graceful worker shutdown; finishing 1 connections
[2022-05-13T08:21:19Z INFO actix_server::accept] Accept thread stopped
[2022-05-13T08:21:39Z INFO shutdown] stopping task
[2022-05-13T08:21:39Z INFO shutdown] task draining items
... never terminates ...
```

## Context
The end goal for me is to implement graceful shutdown of my application that also runs some other tokio `spawn`ed and `spawn_blocking` tasks.

## Your Environment

- Rust Version: rustc 1.60.0 (7737e0b5c 2022-04-04)
- Actix Web Version: 4.0.1
- OS: macOS (12.3.1), Apple M1

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.