unclecode / unclecode/crawl4ai
Feature request: opt-out for the mandatory token when binding non-loopback (Kubernetes / trusted-network deployments)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 83.9k
- Forks
- 8.7k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 11
Description
First off, thanks for the security work in 0.9.0. The secure-by-default direction
is the right call for anyone exposing the server straight to the internet. But the
guard leaves Kubernetes deployments in a genuinely awkward spot, and I wanted to
lay out the situation and ask for a small escape hatch.
After upgrading, the container refuses to boot:
CRITICAL:server:Refusing to start: binding 0.0.0.0 with no CRAWL4AI_API_TOKEN
and jwt_enabled=false would expose an unauthenticated API. Set
CRAWL4AI_API_TOKEN=$(openssl rand -hex 32), enable jwt, or bind loopback.
The suggested "bind to loopback" fallback is a non-starter on Kubernetes. A process
listening on 127.0.0.1 is only reachable inside its own pod's network namespace, so
no other pod and no Service can route to it. Binding to 0.0.0.0 / the pod IP is the
only way to be reachable inside a cluster at all. And the loopback path still mints
an ephemeral token and keeps the auth gate in front of every route, so it doesn't
even yield an unauthenticated service.
That leaves setting a token. I can do that, and I can thread it through my callers.
But in this kind of deployment it buys nothing: the server already lives in a
namespace with default-deny network policies, only a handful of explicitly allowed
workloads can reach it, and nothing outside the cluster can route to it at all. The
trust boundary is enforced at the network layer, which is exactly the model the
platform is built around. Forcing an in-app bearer token on top of that is pure
friction for zero additional security in this topology, and it has to be wired
through every caller and proxy by hand.
What I'd love is a simple, explicit opt-out, something like
security.allow_insecure_bind: true in config.yml or a CRAWL4AI_ALLOW_INSECURE=1
env var, that says "yes, I know what I'm doing, I'm running behind my own trust
boundary" and lets the server bind non-loopback without a token. Keep it off by
default so the secure-by-default promise still holds; it would just be opt-in for
those of us running inside a controlled network.
For now I've had to pin back to the last pre-0.9.0 image to keep things working,
which isn't where I want to stay. A flag like this would let me move onto current
releases cleanly.
Is there an officially supported way to do this that I've missed? If not, would you
be open to adding an opt-out? Happy to send a PR; looks like the check sits right
in _resolve_auth.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the _resolve_auth check mentioned in the issue and trace how non-loopback binding, CRAWL4AI_API_TOKEN, and jwt_enabled are resolved. Determine the supported configuration path for an explicit opt-out, then verify that non-loopback binding without a token works only when opted in and remains protected by default.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, security
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100