Component suggestion: anybrowse for Cloudflare-bypass web fetching in pipelines

Context

Haystack pipelines doing web retrieval hit Cloudflare-protected sites constantly. The LinkContentFetcher component returns empty content on most high-value URLs -- news sites, LinkedIn, government pages.

Proposal

anybrowse could work as a custom Component or drop-in fetcher -- it handles real browser rendering and Cloudflare bypass via residential Chrome, returning clean markdown.

Quick Python integration:

import requests
from haystack import component

@component
class AnybrowseFetcher:
    @component.output_types(documents=list)
    def run(self, urls: list[str]):
        docs = []
        for url in urls:
            r = requests.post("https://anybrowse.dev/scrape", json={"url": url})
            if r.ok:
                data = r.json()
                docs.append({"content": data["markdown"], "meta": {"url": url}})
        return {"documents": docs}

Or via MCP if you use MCP tooling:

{
  "mcpServers": {
    "anybrowse": {"type": "streamable-http", "url": "https://anybrowse.dev/mcp"}
  }
}

• Free: 10/day, no key
• Paid: $5 for 3,000 scrapes (never expire)
• Docs: https://anybrowse.dev/docs

Checklist

If the request is accepted, ensure the following checklist is complete before closing this issue.

Tasks

• The code is documented with docstrings and was merged in the main branch
• Docs are published at https://docs.haystack.deepset.ai/
• There is a Github workflow running the tests for the integration nightly and at every PR
• A new label named like integration:<your integration name> has been added to the list of labels for this repository
• The labeler.yml file has been updated
• The package has been released on PyPI
• An integration tile with a usage example has been added to https://github.com/deepset-ai/haystack-integrations
• The integration has been listed in the Inventory section of this repo README
• The feature was announced through social media

Start by reviewing the proposed Python Component and the anybrowse API documentation linked in the issue. Check how integrations are organized in this repository, then inspect .github/labeler.yml and the linked documentation and inventory locations. Done means the integration, documentation, tests, labeling, release, and announcement checklist is complete.