firecrawl / firecrawl/firecrawl-mcp-server

Custom header support for authentication scenarios

Open
#67 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
7.5k
Forks
884
Avg merge
1d 11h
Merged PRs (30d)
14

Description

I am attempting to use the firecrawl MCP server to scrape a website that requires authentication.

On the firecrawl API, one can specify custom headers like so:

```
curl -X POST https://api.firecrawl.dev/v1/scrape \
-H "Authorization: Bearer YOUR_FIRECRAWL_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://protected.example.com",
"headers": {
"X-Custom-Auth": "token123"
}
}'
```

However, `headers` does not seem to be available as a valid parameter on the [input schema](https://github.com/mendableai/firecrawl-mcp-server/blob/2fee5e697c10fd8be620f6ef1b8d1e886df30476/src/index.ts#L47) for the `firecrawl_scrape` tool, or any other tool, in the MCP server.

I'm almost surprised that this is the case. Is there a work-around or is this on the roadmap to add support for custom headers to scrape requests?

Thank you!

---

Update: I've found that passing the headers object actually works as expected for the scrape command because the implementation [spreads all parameters to the client](https://github.com/mendableai/firecrawl-mcp-server/blob/2fee5e697c10fd8be620f6ef1b8d1e886df30476/src/index.ts#L1108). So, the header just passes through and things work.

```ts
const firecrawlTransport = new StdioClientTransport({
command: "npx",
args: ["-y", "firecrawl-mcp"],
env: {
FIRECRAWL_API_KEY: process.env.FIRECRAWL_API_KEY!,
PATH: process.env.PATH || "/usr/local/bin:/usr/bin:/bin",
},
});
const firecrawlClient = new Client(
{
name: "firecrawl-client",
version: "1.0.0",
},
{
capabilities: {},
}
);
await firecrawlClient.connect(firecrawlTransport);
const result = await firecrawlClient.request(
{
method: "tools/call",
params: {
name: "firecrawl_scrape",
arguments: {
url: `https://example.com`,
formats: ["html"],
onlyMainContent: true,
// Pass the cookie in the headers
headers: {
Authorization: "Bearer ",
},
},
},
},
CallToolResultSchema
);
```

However, this does not work consistently, like for example the extract tool [explicitly re-maps the available options](https://github.com/mendableai/firecrawl-mcp-server/blob/2fee5e697c10fd8be620f6ef1b8d1e886df30476/src/index.ts#L1215C3-L1222C38) and so the headers option is lost.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.