Adopt backend dependency locking (unpinned deps caused the OPTIONS 500 outage, #167)

Summary

backend/requirements.txt leaves most dependencies unpinned (fastapi, anthropic, httpx, pandas, supabase, …), and neither the Docker build (uv pip install -r requirements.txt) nor the Modal build (pip_install_from_requirements) uses a lockfile. Every image build floats to the latest PyPI release at build time.

This directly caused a production outage: FastAPI 0.137.0 (2026-06-14) shipped a breaking routing change, and the next Modal rebuild pulled it with no change on our side, 500-ing every CORS preflight (see #167). A lockfile would have kept the known-good versions and turned that upgrade into a reviewed, CI-gated diff.

Proposal

Introduce a lock so third-party releases can't silently reach production:

• Lighter touch (fits today's layout): keep a loose backend/requirements.in, compile a fully-pinned backend/requirements.txt with uv pip compile requirements.in -o requirements.txt. Direct and transitive deps (FastAPI, the OTel instrumentor) get frozen; upgrades become explicit diffs gated by CI.
• Fuller: move the backend to pyproject.toml + uv.lock (uv sync), updating the Docker / Modal / CI install paths accordingly.

Consideration

redeploy-on-package-update.yml intentionally floats policyengine-uk-compiled to redeploy on new engine releases. Under a lock, that becomes an explicit re-lock + CI step rather than an implicit float — which also stops that cron from silently pulling a broken transitive dependency (exactly what happened here).

Deferred for now (per discussion) — filing so the requirement isn't lost. Related: #167 (hotfix).

Start with backend/requirements.txt, the Docker install command, the Modal pip_install_from_requirements path, and redeploy-on-package-update.yml. Compare the proposed requirements.in compilation and pyproject.toml/uv.lock approaches, including how policyengine-uk-compiled is updated. Done means production installs use reviewed locked dependencies and CI gates dependency changes.