firecrawl / firecrawl/anydoc

Batch mode for bulk directory level processing

Open
#77 0 comments 0 reactions 0 assignees View on GitHub
enhancement P3
Dominant language
Rust
Stars
21.5k
Forks
1.3k
Avg merge
42m
Merged PRs (30d)
17

Description

It would be incredibly helpful if `anydoc` had a built-in feature to process an entire directory of files at once, rather than having to write custom wrapper scripts to handle multiple files.

**Suggested CLI option**:
Possibly a simple --batch or --recursive flag would be perfect for this. For example:
`anydoc --batch ./my-folder -o ./output-folder`

You could also include optional flags like `--keep-ext `(to generate .pdf.md files) and `--passthrough `(to automatically copy standard text files like .txt or .json to the output folder).

## Proposed Behavior
We would love to just point `anydoc` at an input folder and an output folder, and have it do the following:

**1. Keep the folder structure**
Crawl through the input directory (including all subfolders) and recreate that exact same layout in the output directory.

**2. Convert supported documents**
Automatically convert all the supported files (Word, PDF, PowerPoint, etc.) to Markdown. To help us track where the data came from, it should append `.md` to the original name (for example, `handbook.pdf` becomes `handbook.pdf.md`).

**3. Handle CSVs automatically**
Treat CSVs just like the other supported documents and convert them to Markdown automatically, without us needing to pipe them through standard input manually.

**4. Pass through text files**
If it encounters standard plain text files (like `.txt`, `.json`, `.md`, `.py`), just copy them directly to the output folder as-is. Don't convert them or change their names.

**5. Ignore everything else**
If it hits a file it can't support (like images, videos, or zip files), just silently skip it instead of crashing or copying it over.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the anydoc CLI entry point and inspect how input paths, output paths, supported document conversion, and CSV handling currently work. Define the batch or recursive option around the proposed folder behavior: preserve the directory tree, append .md for converted files, pass through named text formats, and skip unsupported files without failing.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.