firecrawl / firecrawl/firecrawl-docs

docs(scrape): document the PDF/document size limit and whether it can be raised

Open Beginner friendly
#1,362 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
MDX
Stars
92
Forks
109
Avg merge
19h 56m
Merged PRs (30d)
113

Description

[Parse](https://docs.firecrawl.dev/features/parse#considerations) documents a 50MB per-request cap. Scrape documents no size limit at all, and the two behave very differently when you cross one.

### What I observed

Same 63MB PDF (a 60MB journal issue), both endpoints:

| endpoint | result | time |
|---|---|---|
| `/v2/parse` | `Uploaded file exceeds maximum size of 50MB` | 9s |
| `/v2/scrape` | `SCRAPE_TIMEOUT` | 4m40s (at `timeout: 280000`) |

A 26MB PDF through `/v2/scrape` parsed in 1.9s, so this is not "big PDFs are slow". I also ruled out a slow origin: `curl` pulled the full 63MB in 4.5s.

So scrape appears to have a size ceiling in the same neighborhood, but instead of a size error you get a timeout after several minutes. In a pipeline that reads as a flaky source, so the natural response is to retry, which costs more time and (per the billing docs, credits are charged whenever infrastructure processes a request) possibly more credits.

### Requests

1. **Document scrape's document size limit.** Even "documents above ~50MB may not complete" would let people plan.
2. **Say whether it can be raised.** My understanding is that this ceiling is account-level rather than a request parameter; there is no setting for it in the dashboard (Overview / Activity Logs / Usage / API Keys / Settings). If raising it is possible, the docs should say who to ask; if not, it should say that too.
3. **Consider a clearer failure than `SCRAPE_TIMEOUT`.** An explicit "document exceeds size limit" would be far more actionable. Filing here since it is a docs repo, happy to move that third point to the main repo if preferred.

### Why it matters

The Parse page tells you the limit up front and fails fast and cheap. Scrape is the endpoint most people point at PDFs, and it currently fails slow and silent.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the Parse considerations page at https://docs.firecrawl.dev/features/parse#considerations and locate the corresponding Scrape documentation. Document the observed document-size limit and explain whether it can be raised, including who to contact or that it cannot be changed. Keep the timeout-behavior request separate if it belongs in the main repository.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.