Run gc.collect() at session teardown to promptly release session-scoped resources

Open
#2,483 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
68/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python

Research direction

Start in shiny/session/_session.py at _run_session_ended_tasks and trace the session teardown path, including how the ExitStack and reactive-domain objects are released. Check the existing session-related tests and verify that cyclic, resource-holding objects are collected promptly after teardown without changing normal session behavior or introducing unacceptable collection overhead.

Written by the indexing model from the issue text.

Description

Summary

When a Shiny session ends, most session-scoped objects are released promptly via refcounting, but a fraction become entangled in reference cycles involving the session machinery (AppSession, reactive domain, callback registries, the ExitStack in _run_impl). Those objects survive until the cyclic GC runs — and because CPython's cyclic collector is triggered by allocation counts rather than wall-clock time, a long-running but idle process may not collect them for hours. For objects holding external resources (HTTP connections, file handles), this means resources outlive their sessions by an unbounded, unpredictable amount of time.

Proposing that Shiny call gc.collect() at the end of session teardown (_run_session_ended_tasks), since session end is a natural, infrequent collection point.

Reproduction

I ran a real Shiny 1.7.0 app whose server function creates an httpx.Client holding a keep-alive connection (the same pattern as creating an LLM client like chatlas.ChatOpenAI() at session start), then drove 120 real websocket sessions through it with the proper init handshake, counting the app process's open TCP connections to the endpoint with lsof:

# app.py
import httpx
from shiny import App, ui

app_ui = ui.page_fluid("fd test")

def server(input, output, session):
    client = httpx.Client()
    client.get("http://127.0.0.1:9876/")  # local HTTP/1.1 keep-alive server

app = App(app_ui, server)

Results:

Stage Open connections in app process
After 30 sessions + 3s settle 4
After 60 total sessions 7
After 100 total sessions 10
After 60s completely idle 10 (unchanged)
After 20 more sessions (new allocations trigger GC) 2

Two things stand out:

  1. ~10% of sessions leave a lingering connection until the cyclic GC runs. All lingering sockets were ESTABLISHED.
  2. An idle process never collects. The count was flat across 60 idle seconds; it only dropped once new sessions created enough allocations to trigger the collector.

Why this matters

This affects any app that creates resource-holding objects per session — HTTP clients (httpx/openai/anthropic SDK clients), database connections, temp files. In a long-running process with bursty traffic (e.g., a Shiny app that's quiet overnight), dead sessions' connections can linger for many hours, holding file descriptors and server-side keep-alive slots. It's not an unbounded leak — collection does eventually happen — but cleanup timing is unpredictable and decoupled from session lifetime.

Explicit close() at session end is the right user-level fix where available, but Shiny can't rely on every object in every server function being explicitly closed.

Proposal

Call gc.collect() at the end of _run_session_ended_tasks() in shiny/session/_session.py. Session ends are infrequent relative to allocations, so the cost should be negligible, and it makes cleanup prompt for all session-scoped cyclic garbage, not just the objects that happen to get explicit teardown.

A lighter-weight alternative would be gc.collect(0) (youngest generation only), though that may not catch cycles involving longer-lived session objects.

Happy to put together a PR if this sounds reasonable.

Dominant language
Python
Stars
1.8k
Forks
135
Avg merge
2d 18h
Merged PRs (30d)
21

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from posit-dev/py-shiny

All issues in posit-dev/py-shiny

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.