firecrawl / firecrawl/firecrawl
Empty catch blocks silently swallow errors across the API (redis, robots.txt, workers)
- Dominant language
- TypeScript
- Stars
- 181k
- Forks
- 9.8k
- Avg merge
- 10h 19m
- Merged PRs (30d)
- 154
Description
## Description
There are a number of empty `catch` blocks in `apps/api` that swallow failures with no logging or metrics, making degradation invisible:
- `apps/api/src/services/redis.ts` — the Redis event handlers (`error`, `reconnecting`, `connect`) wrap their own logging in `try { ... } catch (error) {}`, so a logging failure hides connection-state problems entirely.
- `apps/api/src/controllers/v1/map.ts` — `robots.txt` fetch/import failure during map is silently ignored (`catch (_) {}`), so maps may ignore robots rules without any trace.
- `apps/api/src/controllers/v0/crawl.ts` — same silent robots.txt swallow.
- `apps/api/src/lib/crawl-redis.ts` — `crawler.importRobotsTxt(sc.robots)` failures are dropped.
- `apps/api/src/services/worker/scrape-worker.ts` — a `catch (e) {}` around result handling drops errors when returning a completed job's document.
- `apps/api/src/scraper/scrapeURL/lib/smartScrape.ts` — JSON parse of an error response is swallowed (`catch (e) {}`), potentially masking cost-limit errors.
## Suggested fix
At minimum, add `logger.debug`/`logger.warn` (with context) and, where relevant, a metric to each of these sites so silent degradation becomes observable. For the robots.txt paths, a warn-level log seems appropriate since behavior changes (rules not applied) when the fetch fails.
Contributor guide
Research direction
Review the six locations named in apps/api, starting with the Redis handlers in services/redis.ts and the robots.txt paths in controllers/v1/map.ts, controllers/v0/crawl.ts, and lib/crawl-redis.ts. Then inspect the empty catches in scrape-worker.ts and smartScrape.ts. Done means each failure is observable through appropriately contextualized debug or warn logging and, where relevant, a metric, without silently swallowing degradation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100