junaid33 / junaid33/opensource.builders

Add fastCRW (open-source alternative to Firecrawl and Tavily)

Open
#560 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
1.3k
Forks
193
PR merge metrics
No merged PRs in 30d

Description

Add open source tool: fastCRW
- Repository: https://github.com/us/crw
- Website: https://fastcrw.com
- Description: Open-source web scraping, crawling, AI search, and data extraction API in a single 8 MB Rust binary. Firecrawl-compatible /v1 endpoint and Tavily-compatible search responses. 710 ms p50 on a 1,000-URL benchmark, 6.6 MB idle RAM.
- Category: API Development
- Features: Self-hosted option, API integration, Markdown support, Plugin system, Data export

## Open source alternative to

1. Firecrawl - https://firecrawl.dev
2. Tavily - https://tavily.com

## Additional info

- License: AGPL-3.0
- Primary language: Rust
- Endpoints: scrape, crawl, search, extract, map (Firecrawl /v1 compatible)
- Search response format: Tavily-compatible
- Free tier: 500 credits/month on managed cloud
- Self-host: AGPL-3.0, single 8 MB Docker image
- SDKs: Python, Node.js, CLI
- Integrations: LangChain, LlamaIndex, CrewAI, n8n, Dify
- Distribution: Homebrew, apt, Docker

## Benchmark

On a public 1,000-URL benchmark fastCRW posts 710 ms p50 and 833 ms average per URL, runs in 6.6 MB idle RAM, and ships as an 8 MB Docker image.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.