Whitelist Kiwix workers from Cloudflare Anti-Bot protection

We are more and more impacted by Cloudflare anti-bot protections. Looks like these protections are mostly targeting AI-bots, see https://github.com/openzim/mwoffliner/issues/2134#issuecomment-2671525401

Also see https://radar.cloudflare.com/traffic/verified-bots


I tried to request for a bot whitelisting from my own Cloudflare account and this is the form to fill:


I'm sure we should apply to this for our Zimfarm workers.

**Prerequisite is obviously that scrapers properly honor robots.txt correctly** (none do AFAIK) **and use proper User-Agent** (most do)

I think applying for only one Bot for all our tools is sufficient, Cloudflare does not want to dig into our implementation details, and maintaining to many things on their side is going to hurt.

Start with Cloudflare's verified-bots information and the bot-whitelisting form linked in the issue. Confirm how the Zimfarm workers identify themselves and whether they honor robots.txt; the work is done when Cloudflare approves a suitable whitelist for the workers.

Written by the indexing model from the issue text.