pocket-ic-server leaks child processes when add_instance panics mid-setup

Summary

When the create_instance closure in add_instance (rs/pocket_ic_server/src/state_api/state.rs around L626) panics after partially constructing a PocketIc, the pocket-ic child processes it has already spawned are not cleaned up. They accumulate until the host's process/thread table is exhausted, at which point the replica is permanently broken and needs a manual container/host restart.

(Filing here since dfinity/ic has issues disabled — happy to move if there's a better home.)

Observed

A Synology Docker container running dfx start (dfx 0.31.0, pocket-ic under the hood) accumulated 15,783 orphaned pocket-ic processes over ~2 days of uptime. Container logs showed a repeating pattern:

thread 'tokio-runtime-worker' panicked at rs/pocket_ic_server/src/state_api/state.rs:626:14:
Failed to create PocketIC instance: JoinError::Panic(Id(21), "failed to spawn thread: Os { code: 11, kind: WouldBlock, message: \"Resource temporarily unavailable\" }", ...)
ERROR: Failed to initialize PocketIC: error sending request for url (http://localhost:XXXXX/instances): client error (SendRequest): connection closed before message completed

Once something transient caused the first thread-spawn failure, each subsequent add_instance call leaked another pocket-ic child, which in turn consumed more kernel threads, making the next thread-spawn even more likely to fail. Runaway growth.

Root cause — two contributing factors

1. add_instance doesn't catch panics from create_instance

let instance = tokio::task::spawn_blocking(move || create_instance(seed, gateway_port))
    .await
    .expect("Failed to create PocketIC instance")?;

.expect() on JoinError re-panics in the calling task. Anything create_instance already spawned (child processes, tokio tasks, OS threads) before hitting its own panic is dropped during unwind as local variables.

2. No Drop impl that cleans up child processes

Searching state_api/state.rs there's an impl Drop for HttpGateway { ... } but no impl Drop for PocketIc, Instance, or InstanceState. Unwinding drops the partial PocketIc value, but std::process::Child::drop does not kill the child (Rust's well-known detach-on-drop), and thread::JoinHandle::drop just detaches. So the pocket-ic child processes keep running, indefinitely, orphaned.

Suggested fix

Primary — impl Drop for PocketIc

The instance struct has to take active responsibility for cleanup. Approximately:

impl Drop for PocketIc {
    fn drop(&mut self) {
        for child in self.child_processes.iter_mut() {
            let _ = child.kill();
            let _ = child.wait(); // reap to avoid zombies
        }
        for handle in self.tokio_handles.drain(..) {
            handle.abort();
        }
    }
}

Belt-and-braces — kill_on_drop(true)

Spawn pocket-ic children via tokio::process::Command::new("pocket-ic").kill_on_drop(true) instead of std::process::Command. Partial-setup panics during construction then clean themselves up via unwind without needing a custom Drop.

Defensive — catch panics in add_instance

Wrap the closure in std::panic::catch_unwind(AssertUnwindSafe(...)) so a panic becomes an Err return rather than propagating:

let instance = tokio::task::spawn_blocking(move || {
    std::panic::catch_unwind(std::panic::AssertUnwindSafe(
        || create_instance(seed, gateway_port)
    ))
}).await
    .map_err(|e| format!("join error: {e}"))?
    .map_err(|_| "create_instance panicked during setup".to_string())??;

Combined with a proper Drop impl this gives clean error propagation without leaks, regardless of where inside create_instance the failure occurs.

Reproducer

1. Set a host (or container cgroup) ulimit -u low enough that spawning ~10 OS threads hits the limit.
2. Send repeated POST /instances requests to a running pocket-ic-server.
3. Observe ps -eo comm | grep -c pocket-ic growing with each failed call.
4. The orphaned processes survive indefinitely with no way for the server to reclaim them.

Impact

Makes dfx start unreliable on resource-constrained hosts (small VMs, NAS devices, containers with pid limits). Once the thread table is exhausted, every new instance creation fails, and every failure compounds the problem. Only a container/host restart recovers the system; there is no in-process mechanism to reap the leaked pocket-ic processes.

Environment

• dfx: 0.31.0
• OS: Synology DSM (Linux), running pocket-ic-server inside a Docker container
• Architecture: x86_64 (Intel Xeon D-1527)
• Host: 16GB RAM, host-wide ulimit -u 62049 — the limit hit is kernel-level thread allocation, not the user rlimit

Start in rs/pocket_ic_server/src/state_api/state.rs around add_instance and the create_instance closure near line 626; trace how child processes are created and what is dropped when setup panics. Reproduce with repeated POST /instances requests under a low thread limit, then verify failed setup reclaims child processes and does not cause pocket-ic process counts to grow.