firecrawl / firecrawl/open-lovable

Firecrawl API Returns "Bad Request" Error Due to Invalid URL in `/api/scrape-screenshot` Route

Open
#8 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
28.4k
Forks
5.4k
PR merge metrics
No merged PRs in 30d

Description

### Firecrawl API Returns "Bad Request" Error Due to Invalid URL in `/api/scrape-screenshot` Route

#### Description
When running the Open Lovable app locally after setup, attempting to use the screenshot/scrape functionality results in a 500 error from the `/api/scrape-screenshot` endpoint. The error indicates a "Bad Request" from the Firecrawl API, specifically complaining about an invalid URL (missing valid top-level domain or invalid path).

This seems to occur because the URL being passed to Firecrawl is not a valid, publicly accessible one—possibly a local URL like `http://localhost:3000` or an improperly formed E2B sandbox URL.

#### Steps to Reproduce
1. Clone the repo: `git clone https://github.com/mendableai/open-lovable.git`
2. Install dependencies: `cd open-lovable && npm install`
3. Create `.env.local` with required keys:
```
E2B_API_KEY=your_e2b_key # Obtained from https://e2b.dev
FIRECRAWL_API_KEY=your_firecrawl_key # Obtained from https://firecrawl.dev
# Optional AI key, e.g., ANTHROPIC_API_KEY=your_anthropic_key
```
4. Run the app: `npm run dev`
5. Open http://localhost:3000 and interact with the chat to build a React app (which triggers the scrape/screenshot).
6. Observe the error in the console/server logs:
```
Error: Firecrawl API error: {"success":false,"error":"Bad Request","details":[{"validation":"url","code":"invalid_string","message":"Invalid url","path":["url"]},{"code":"custom","message":"URL must have a valid top-level domain or be a valid path","path":["url"]},{"code":"custom","message":"Invalid URL","path":["url"]}]}
at POST (app/api/scrape-screenshot/route.ts:35:12)
POST /api/scrape-screenshot 500 in 567ms
```

#### Expected Behavior
The app should successfully scrape/screenshot the generated React app preview (likely from an E2B sandbox) without Firecrawl errors.

#### Actual Behavior
Firecrawl rejects the request with a "Bad Request" due to an invalid URL.

#### Additional Details
- **Environment**:
- Node.js version: v23.9.0
- OS: macOS 15.3.2 (24D81)
- Browser: Brave 1.80.125
Chromium: 138.0.7204.184
- **Troubleshooting Tried**:
- Verified Firecrawl API key works directly via curl:
```
curl -X POST https://api.firecrawl.dev/v0/scrape \
-H "Authorization: Bearer fc-XXXX" \
-H "Content-Type: application/json" \
-d '{"url": "https://www.google.com"}'
```
This returns a successful response with scraped content.
- Added logging to `app/api/scrape-screenshot/route.ts` to check the URL being sent – it appears to be invalid (e.g., local or malformed).
- Ensured E2B_API_KEY is set, as it's required for generating sandbox previews.
- **Possible Causes**:
- The URL generated for the preview (e.g., from E2B sandbox) might not be fully qualified or publicly accessible.
- If E2B sandbox fails to initialize, it could fallback to a local URL, which Firecrawl can't access.
- Lack of validation in the route for the URL before sending to Firecrawl.

Image

#### Suggested Fix
- Add URL validation in `app/api/scrape-screenshot/route.ts` before the Firecrawl fetch (e.g., using `valid-url` package).
- Ensure the preview URL is always a public E2B sandbox URL; if local, perhaps integrate a tunneling service like ngrok for development.
- Hardcode a test URL in the route temporarily to confirm it works with valid inputs.

This blocks local development for new users. Happy to provide more logs or test patches!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.