unclecode / unclecode/crawl4ai

ManagedBrowser SEGV_ACCERR crash with --disable-features=OptimizationHints in --headless=new (macOS arm64)

Open Beginner friendly
#2,239 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

⚙ Done
Dominant language
Python
Stars
83.9k
Forks
8.7k
Avg merge
3d 7h
Merged PRs (30d)
11

Description

Bug

ManagedBrowser's hardcoded --disable-features=OptimizationHints,MediaRouter,DialMediaRouteProvider flag crashes Chrome-for-Testing with a SEGV_ACCERR segfault immediately after the CDP endpoint comes up, when launched with --headless=new. crawl4ai reports this generically as "CDP endpoint not ready after startup", which gives no indication a crash (rather than a slow startup) occurred.

Environment

  • crawl4ai version: 0.9.3
  • OS: macOS 26.6.2 (Darwin, arm64 / Apple Silicon)
  • Browser: Chrome-for-Testing 151.0.7922.34
  • Launch config: BrowserConfig(headless=True, use_managed_browser=True, user_data_dir=<persistent profile>) — i.e. the ManagedBrowser code path (raw subprocess + CDP connect), not the Playwright-managed BrowserManager path.

Steps to reproduce

  1. Launch Chrome-for-Testing directly with the exact flag list ManagedBrowser.build_browser_flags constructs, including --headless=new and --disable-features=OptimizationHints,MediaRouter,DialMediaRouteProvider.
  2. Redirect stdout/stderr to a log file and poll http://localhost:<port>/json/version for CDP readiness.
  3. Observe: the log shows DevTools listening on ws://... (CDP genuinely came up), immediately followed by Received signal 11 SEGV_ACCERR and a crash stack trace. The process then dies, so the 5-attempt exponential-backoff readiness poll in _verify_cdp_ready() eventually times out and crawl4ai surfaces only "CDP endpoint not ready after startup".

Isolating the cause

I bisected the ~20-flag launch argument list by binary search (removing half the flags, reproducing/not-reproducing, repeating) using direct manual Chrome launches outside of crawl4ai. This isolated the crash to a single flag:

  • --disable-features=OptimizationHints,MediaRouter,DialMediaRouteProvidercrashes
  • --disable-features=MediaRouter,DialMediaRouteProvider (i.e. OptimizationHints removed) → does not crash
  • MediaRouter and DialMediaRouteProvider are safe both individually and combined; OptimizationHints alone in the disable-list reproduces the crash by itself.

This was not a stale-lock-file or --disable-gpu-related issue — both of those were ruled out first (removed Singleton{Lock,Cookie,Socket}, confirmed no stale process held them via ps -p <pid>; removed --disable-gpu and still reproduced the crash). The only thing that changed the outcome was the presence/absence of OptimizationHints in --disable-features.

Fix applied locally

Removed OptimizationHints from both --disable-features flag lists in crawl4ai/browser_manager.py (ManagedBrowser.build_browser_flags, and a sibling list further down the same file), leaving MediaRouter,DialMediaRouteProvider. This fully resolves the crash on this machine with no other observed regressions.

Possibly related but distinct

#1445 ("Identity based browsing fails") looks superficially similar (CDP/managed-browser startup failure) but its root cause there was diagnosed as a CDP-readiness-check timing race (asyncio.sleep waits too short before Chrome fully initializes), not a crash — the browser process in that report doesn't appear to actually die. This issue is a genuine SEGV crash triggered by one specific Chromium feature-disable flag, reproducible outside of crawl4ai entirely via a direct Chrome launch, so I believe it's a separate root cause worth tracking on its own even though the surface symptom (CDP never becomes ready) looks similar.

Suggested fix

Drop OptimizationHints from the --disable-features list in ManagedBrowser.build_browser_flags (and the sibling list), or make the flag list configurable so users hitting Chromium-version/platform-specific crashes on a particular feature flag can override it without patching the package.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in crawl4ai/browser_manager.py at ManagedBrowser.build_browser_flags and the sibling --disable-features list identified in the issue. Remove OptimizationHints from both lists, then exercise the managed-browser startup path with headless=new and verify that Chrome remains running and the CDP readiness check succeeds.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.