firecrawl / firecrawl/cli

firecrawl agent silent stdout failure when --max-credits insufficient is a dangerous category bug for agentic consumers

Open
#115 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
631
Forks
100
Avg merge
5h 9m
Merged PRs (30d)
25

Description

## Summary (the category)

`firecrawl agent` exhibits a dangerous failure mode for agentic consumers: when `--max-credits` is insufficient to complete the query, the command returns **0 bytes on stdout** while writing the failure message only to stderr. This is the canonical "silent corruption" pattern that breaks downstream pipelines.

This issue is about the **failure-mode category**, not the specific credit threshold.

## Repro

`firecrawl-cli==1.18.0`, macOS 25.5.0, paid account (51K+ credits available):

```
$ firecrawl agent "How does Andrej Karpathy use Obsidian for note-taking?" \
--model spark-1-mini --max-credits 50 --wait \
> out.txt 2> err.txt
$ wc -c out.txt err.txt
0 out.txt # <-- silent failure
~118 KB err.txt # <-- spinner spam (separate concern, see #102)

$ grep -v '^\[K[⠋⠙⠹⠸⠼⠴⠦⠧⠇⠏]' err.txt | tail
✗ Agent failed
Error: Refusal: Error: Agent reached max credits

$ echo $? # exit code
0 # <-- ALSO 0, so `set -e` callers won't notice
```

Bumping `--max-credits` to 200 makes the same query succeed (returns ~2 KB of content).

## Why this is dangerous

A downstream agentic consumer capturing stdout sees:
- Empty result (looks like "agent found nothing")
- Zero exit code (looks like success)

That consumer will proceed with empty input and produce a silent-corruption error chain. In our case (multi-vendor scatter-gather research workflow), the FireCrawl subagent silently produced no content for the synthesizer, and we only discovered the failure when comparing per-vendor file sizes against expectations.

This is a **category** of bug — any agentic consumer that exhausts credits at any model/query combo will hit it.

## Proposed fixes (any one is sufficient)

| Option | Effort | Benefit |
|---|---|---|
| **A**. Exit non-zero on max-credits exhaustion | Low | `set -e` and shell callers detect it; doesn't change output shape |
| **B**. Mirror the error to stdout as a JSON envelope: `{"error": "max_credits_exceeded", "message": "...", "job_id": "..."}` | Medium | Agentic consumers parsing stdout JSON detect it |
| **C**. Document a `--max-credits` floor per model in `--help` (e.g., "spark-1-mini: 200 minimum") | Low | Doc-only; doesn't fix the silent-failure category but prevents misuse |

Option A is the cheapest + most defensive. Option B is the most consumer-friendly (especially when paired with #102's `--json` work). Option C is doc-only and complementary.

## Related

- #102 ("Improve CLI outputs for agentic use") proposes `--quiet/--silent/--concise`. A `--strict` mode that escalates stderr errors to non-zero exit + stdout JSON envelope would directly address this.
- The spinner spam mentioned above is a separate concern; the silent-stdout-failure can be fixed independently.

## Repro environment

- firecrawl-cli: 1.18.0 (installed via `npm install -g firecrawl-cli`)
- OS: macOS 25.5.0 (Darwin), zsh
- Auth: stored credentials (post `firecrawl login --api-key`)
- Account: paid plan, 51,238 credits available — so it's not a "ran out of account credits" symptom, it's strictly a `--max-credits` budget cap
- Model tested: `spark-1-mini` (failed at `--max-credits 50`, succeeded at 200)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.