unclecode/crawl4ai

View on GitHub

🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN

Stars: 83.9k
Forks: 8.7k
Open beginner issues: 0
Indexed issues: 43
Avg merge: 3d 7h
Merged PRs (30d): 11

Dominant language: Python
License: Apache-2.0
Last GitHub push: Aug 31, 2026
Latest indexed: Sep 20, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

43 open issues indexed

[Bug]: Playground Advanced Config panel is non-functional — still sends the removed `code` protocol to /config/dump Open

⚙ Done 🐞 Bug 🐳 Docker 📌 Root caused

Difficulty 4/5 3-5 days Newbie friendliness 52/100

unclecode/crawl4ai#2260 · 1 comment · Opened Sep 14, 2026
[Bug]: DefaultTableExtraction drops <th> row headers and ignores rowspan, producing misaligned result.tables Open

⚙ Done 🐞 Bug

Difficulty 3/5 1-2 days Newbie friendliness 78/100

unclecode/crawl4ai#2258 · 1 comment · Opened Sep 13, 2026
Discord GitHub Notifications fails for every new stargazer Open

⚙️ In-progress 🐞 Bug 📌 Root caused

Difficulty 3/5 1-2 days Newbie friendliness 58/100

unclecode/crawl4ai#2255 · 2 comments · Opened Sep 13, 2026
[docs] DOM XSS: unescaped innerHTML interpolation in docs/apps/linkdin graph view template (5 sinks) Open

🐞 Bug 📌 Root caused

Difficulty 4/5 3-5 days Newbie friendliness 68/100

unclecode/crawl4ai#2251 · 5 comments · Opened Sep 12, 2026
[Bug]: BFSDeepCrawlStrategy re-scans the full level for parent lookup (O(n²)); BestFirstCrawlingStrategy can enqueue the same URL twice Open

⚙ Done 🐞 Bug

Difficulty 4/5 3-5 days Newbie friendliness 55/100

unclecode/crawl4ai#2242 · 2 comments · Opened Sep 9, 2026
ManagedBrowser SEGV_ACCERR crash with --disable-features=OptimizationHints in --headless=new (macOS arm64) Open

⚙ Done

Difficulty 2/5 1-3 hours Newbie friendliness 88/100

unclecode/crawl4ai#2239 · 1 comment · Opened Sep 8, 2026
[Bug]: browser_mode=builtin fails with "CDP endpoint not ready" — Chromium segfaults (SIGSEGV) when spawned from gunicorn worker (v0.9.3) Open

⚙️ In-progress 🐞 Bug

Difficulty 4/5 3-5 days Newbie friendliness 48/100

unclecode/crawl4ai#2238 · 1 comment · Opened Sep 8, 2026
[Bug]: MCP SSE transport rejects concurrent requests on connect with "Received request before initialization was complete" (-32602) Open

⚙️ In-progress 🐞 Bug 📌 Root caused

Difficulty 4/5 3-5 days Newbie friendliness 66/100

unclecode/crawl4ai#2233 · Opened Sep 7, 2026
[Bug]: Pooled browsers get ~5 s/navigation slower with use, and the janitor never recycles a busy pool Open

⚙️ In-progress 🐞 Bug

Difficulty 4/5 3-5 days Newbie friendliness 42/100

unclecode/crawl4ai#2231 · 1 comment · Opened Sep 6, 2026
[Bug]: Disallow: /*? in robots.txt blocks the whole website. Open

⚙️ In-progress 🐞 Bug

Difficulty 3/5 1-2 days Newbie friendliness 72/100

unclecode/crawl4ai#2225 · 2 comments · Opened Sep 1, 2026
[Bug]: Playground: /md and /llm endpoints always fail because Advanced Config `code` is sent to /config/dump Open

⚙️ In-progress 🐞 Bug

Difficulty 2/5 1-3 hours Newbie friendliness 86/100

unclecode/crawl4ai#2222 · 2 comments · Opened Aug 31, 2026
Clarify supported versions in SECURITY.md after v0.9.2 release Open

⚙️ In-progress

Difficulty 2/5 1-3 hours Newbie friendliness 58/100

unclecode/crawl4ai#2216 · 1 comment · Opened Aug 30, 2026
page_timeout is silently capped at 60s over HTTP with no way to raise it Open

⚙️ In-progress 🐞 Bug 📌 Root caused

Difficulty 4/5 3-5 days Newbie friendliness 58/100

unclecode/crawl4ai#2211 · 1 comment · Opened Aug 28, 2026
[Bug]: MCP "ask" tool exposes an empty parameters schema ({"type":"object"} with no "properties"), rejected by strict tool-calling clients Open

🐞 Bug 🩺 Needs Triage

Difficulty 3/5 1-2 days Newbie friendliness 72/100

unclecode/crawl4ai#2207 · 1 comment · Opened Aug 27, 2026
[Bug]: Post-navigation page operations have no timeout, and page cleanup is skipped on task cancellation — pages can leak and crawls can hang forever Open

⚙️ In-progress 🐞 Bug

Difficulty 4/5 3-5 days Newbie friendliness 52/100

unclecode/crawl4ai#2205 · Opened Aug 27, 2026
[Bug]: Docker pool's permanent browser never serves any request - signature is computed without the egress proxy, so no `/crawl` can ever match it Open

⚙️ In-progress 🐞 Bug

Difficulty 3/5 1-2 days Newbie friendliness 74/100

unclecode/crawl4ai#2204 · 1 comment · Opened Aug 27, 2026
[Bug]: Browser pages and renderer processes accumulate for weeks on non-streaming /crawl (Docker 0.9.2) Open

⚙️ In-progress 📌 Root caused

Difficulty 4/5 3-5 days Newbie friendliness 48/100

unclecode/crawl4ai#2202 · 3 comments · Opened Aug 26, 2026
[Bug]: Docker server fails to boot after mcp 2.0.0 release — mcp_bridge uses the removed v1 low-level API Open

Difficulty 1/5 Under an hour Newbie friendliness 84/100

unclecode/crawl4ai#2147 · 3 comments · Opened Aug 18, 2026
Feature request: opt-in block_internal_urls egress filter for the library layer Open

✨ Enhancement

Difficulty 5/5 Over a week Newbie friendliness 45/100

unclecode/crawl4ai#2146 · 2 comments · Opened Aug 17, 2026
[Bug]: preserve_tags/preserve_classes are a no-op for excluded tags (aside, nav, footer, header, form) Open

🐞 Bug 🩺 Needs Triage

Difficulty 3/5 1-2 days Newbie friendliness 78/100

unclecode/crawl4ai#2125 · 2 comments · Opened Aug 6, 2026
[Bug]: Docker memory guard reads host RAM when no container limit is set — cgroup v2 "max" defeats get_container_memory_percent Open

Difficulty 2/5 1-3 hours Newbie friendliness 72/100

unclecode/crawl4ai#2123 · 1 comment · Opened Aug 5, 2026
[Bug]: crawler.base_config boolean values are silently ignored (regression from #1505) Open

🐞 Bug 🩺 Needs Triage

Difficulty 4/5 3-5 days Newbie friendliness 55/100

unclecode/crawl4ai#2121 · 1 comment · Opened Aug 5, 2026
[Bug]: A timeout occurs when attempting to connect via MCP from LM Studio. Open

🐞 Bug 🩺 Needs Triage

Difficulty 4/5 3-5 days Newbie friendliness 35/100

unclecode/crawl4ai#2120 · 1 comment · Opened Aug 4, 2026
PruningContentFilter drops whitespace-only spans in <pre>/<code> — #1181's bug, still present on the fit_markdown path Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

unclecode/crawl4ai#2110 · 2 comments · Opened Jul 30, 2026
unclecode-litellm shadows the upstream litellm package; the quarantine that motivated the fork has been lifted Open

Difficulty 3/5 1-2 days Newbie friendliness 52/100

unclecode/crawl4ai#2098 · 1 comment · Opened Jul 25, 2026
[Bug]: arm64 image ships ~2.4GB of build-time waste (core dumps + duplicate Chromium cache), root-caused to QEMU CI emulation Open

⚙️ In-progress 🐞 Bug

Difficulty 3/5 1-2 days Newbie friendliness 62/100

unclecode/crawl4ai#2092 · 1 comment · Opened Jul 22, 2026
[Bug]: AsyncWebCrawler.arun_many() stream closure does not await dispatcher cleanup in 0.9.2 Open

🐞 Bug 🩺 Needs Triage

Difficulty 3/5 1-2 days Newbie friendliness 72/100

unclecode/crawl4ai#2083 · 1 comment · Opened Jul 16, 2026
[Bug]: BFSDeepCrawlStrategy.can_process_url() rejects valid single-label hostnames (netloc without a dot) Open

🐞 Bug 🩺 Needs Triage

Difficulty 3/5 1-2 days Newbie friendliness 68/100

unclecode/crawl4ai#2079 · 1 comment · Opened Jul 15, 2026
Config option to disable is_blocked() post-crawl content veto Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

unclecode/crawl4ai#2058 · 2 comments · Opened Jul 8, 2026
Add Circle/x402 paid crawl endpoint for autonomous agents Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

unclecode/crawl4ai#2024 · Opened Jun 20, 2026
Feature request: opt-out for the mandatory token when binding non-loopback (Kubernetes / trusted-network deployments) Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

unclecode/crawl4ai#2023 · 2 comments · 1 reaction · Opened Jun 19, 2026
Feature Request: Auto-Follow Single-Frame Framesets Open

✨ Enhancement

Difficulty 4/5 3-5 days Newbie friendliness 55/100

unclecode/crawl4ai#1974 · 1 comment · Opened May 20, 2026
[Bug]: Markdown export loses heading hierarchy and table structure Open

❓ Question

Difficulty 4/5 3-5 days Newbie friendliness 42/100

unclecode/crawl4ai#1964 · 3 comments · Opened May 11, 2026
[Bug]: MCP scrape tools lack wait_until / SPA support that REST API and CLI provide Open

⚙️ In-progress 🐞 Bug 📌 Root caused

Difficulty 4/5 3-5 days Newbie friendliness 52/100

unclecode/crawl4ai#1963 · 2 comments · Opened May 11, 2026
📝 Integration Proposal: CAJAL — Scientific Paper Agent for Crawl4AI Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

unclecode/crawl4ai#1956 · Opened May 4, 2026
[Bug]: Browser hangs indefinitely on WSL when system proxy is required Open

❓ Question

Difficulty 4/5 3-5 days Newbie friendliness 45/100

unclecode/crawl4ai#1930 · 3 comments · Opened Apr 22, 2026
[Bug]: Not Compatible with Python 3.14 (lxml dependency failure) Open

⚰️ Wontfix

Difficulty 3/5 1-2 days Newbie friendliness 52/100

unclecode/crawl4ai#1903 · 2 comments · 4 reactions · Opened Apr 6, 2026
[Bug]: `downloaded_files` race condition causes cross-contamination between CrawlResults Open

⚙️ In-progress 🐞 Bug

Difficulty 4/5 3-5 days Newbie friendliness 45/100

unclecode/crawl4ai#1889 · 1 comment · Opened Mar 31, 2026
[Bug]: LLMExtractionStrategy not applied when CrawlerRunConfig.cache_mode=ENABLED Open

✊🏼 On-hold 🐞 Bug 📌 Root caused

Difficulty 3/5 1-2 days Newbie friendliness 52/100

unclecode/crawl4ai#1455 · 2 comments · Opened Aug 29, 2025
Bring Docker API server to feature parity with Crawl4AI v0.7.x Open

⚙️ Under Review ✨ Enhancement

Difficulty 5/5 Over a week Newbie friendliness 25/100

unclecode/crawl4ai#1452 · 3 comments · 4 reactions · Opened Aug 28, 2025
[Bug]: Memory Leak on Repeated /md Requests via Docker (MacOS) — Container Crashes Randomly Over Time Open

❓ Question 🐳 Docker

Difficulty 4/5 3-5 days Newbie friendliness 45/100

unclecode/crawl4ai#1256 · 15 comments · 10 reactions · Opened Jun 29, 2025
[Bug]: RateLimiter provides ineffective protection against failures Open

⚙️ In-progress 🐞 Bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

unclecode/crawl4ai#1095 · 2 comments · Opened May 10, 2025
[Bug]: When extracting data with scroll_full_page, only the final elements get parsed Open

⚙️ In-progress 🐞 Bug

Difficulty 4/5 3-5 days Newbie friendliness 47/100

unclecode/crawl4ai#731 · 22 comments · 3 reactions · Opened Feb 20, 2025

unclecode/crawl4ai

Get new issues in your inbox