[Bug]: Browser pages and renderer processes accumulate for weeks on non-streaming /crawl (Docker 0.9.2)

crawl4ai version

0.9.2 (Docker image unclecode/crawl4ai:0.9.2, C4AI_VERSION=0.9.2)

Expected Behavior

After a non-streaming POST /crawl completes, the pages and browser contexts it opened are closed. A container that has served no traffic for two weeks sits at near-zero CPU and holds only the warm browser pool.

Current Behavior

The container had been up 29 days serving a low volume of non-streaming /crawl requests (a handful per day, max_depth=2, at most 5 pages). At inspection it held 46 chromium processes and had not served a request for 15 days:

 3  chrome-headless-shell (browser root)
 2  playwright/driver/node
35  chrome-headless-shell --type=renderer
 4  --type=zygote
 3  --type=utility
 2  --type=gpu

945 MB RSS across those processes, 957 MiB for the container, and a steady 31% of a CPU core with zero traffic — two consecutive docker stats samples read 30.69% and 30.86%.

Renderer start timestamps cluster on the days crawls actually ran: dozens on Jul 29 between 19:41 and 23:21, isolated ones later, the newest on Aug 11. A --type=renderer process lives exactly as long as its page, so those pages were never closed and the renderers outlived their requests by weeks.

There are also two independent driver → browser root branches, started Jul 28 14:25:55 and Jul 29 17:21:15. The older branch was never torn down when the newer one started.

Restarting the container dropped it to 7 chromium processes, 426 MiB and 0.4% CPU.

Is this reproducible?

Yes

Inputs Causing the Bug

Plain non-streaming request — no session id, no hooks:

{
  "urls": ["<single http(s) url>"],
  "browser_config": {"type": "BrowserConfig", "params": {"headless": true, "accept_downloads": false}},
  "crawler_config": {"type": "CrawlerRunConfig", "params": {
    "stream": false,
    "cache_mode": "bypass",
    "wait_until": "domcontentloaded",
    "page_timeout": 30000,
    "process_iframes": false,
    "remove_overlay_elements": true,
    "exclude_external_images": true
  }}
}

Steps to Reproduce

1. Run unclecode/crawl4ai:0.9.2 in Docker with no memory or CPU limit on a cgroup v2 host.
2. Send the request above a few times a day for a few days, then stop sending traffic.
3. After some days, inspect the process table inside the container.

OS

Linux, Docker, cgroup v2 host

Python version

3.12.13 (image default)

Browser

chromium_headless_shell-1228, bundled with the image

Error logs & Screenshots

Nothing is logged and the container reports healthy the whole time.

Possibly related: #943 (closed, fixed in 0.8.5), #2083, #2155, #2123. This container had no memory limit, so per #2123 the memory guard reads host RAM instead of container usage — that may be why nothing reclaims these pages.

Start by reproducing the non-streaming POST /crawl request in the Docker 0.9.2 image, then trace its browser, context, and page cleanup lifecycle. Compare the behavior with the related issues cited in the report. Done means completed requests leave no accumulated pages or renderer processes and do not create lingering browser-root branches.