CI stability audit: unpinned/floating dependencies that can flip CI from green to red

Summary

Audit (verified line-by-line against 9627f82c60e9ffb7d51dc7e6277cf4d0f61f01b4) of everything in this repo whose resolved behavior can change while the git SHA under test stays exactly the same — unpinned package versions, floating MCP/CLI invocations, mutable Action refs, unpinned Docker/OS inputs. Any of these can turn a previously-green commit red (or silently change agent behavior) with zero code changes on our side.

Verification pass corrected one thing from the original draft: the "moving major tag" claim for astral-sh/setup-uv / tj-actions/changed-files only applies to 3 specific stray call sites — the other ~24 occurrences of those actions are already correctly SHA-pinned (@<40-char-sha> # vN). That's called out below so we don't waste effort re-pinning things that are already fine.

Immediate risks (active in default/CI paths today)

• openhands-tools/.../subagents/web_researcher.md:12 — uvx mcp-server-fetch, unpinned, in an active subagent preset.
• openhands-tools/.../browser_use/impl.py:238 — runtime auto-install runs uvx playwright install chromium --with-deps --no-shell, unpinned.
• .github/workflows/tests.yml:230 — CI separately runs unpinned uvx playwright install chromium.
• openhands-agent-server/.../docker/Dockerfile:182-185 — npm install -g @agentclientprotocol/claude-agent-acp@0.44.0 @agentclientprotocol/codex-acp@1.1.2 @google/gemini-cli@0.46.0: top-level packages are exact-pinned, but there's no lockfile for this ad-hoc install, so every transitive npm dependency resolves fresh at build time.
• openhands-sdk/.../skills/skill.py:1121-1140 — public-skills loader defaults PUBLIC_SKILLS_REF to "main" on OpenHands/extensions, and re-fetches every 60s while public skills are enabled. (Mitigation exists: EXTENSIONS_REF env var can pin to a full SHA — it's just not the default.)
• openhands-sdk/.../critic/impl/api/chat_template.py:63 — critic fetches tokenizer config from https://huggingface.co/{name}/raw/main/tokenizer_config.json on cache miss.

CI workflows: unpinned/unlocked installs

• .github/workflows/qa-changes-evaluation.yml:66 — pip install lmnr, unpinned.
• .github/workflows/version-bump-prs.yml:246 — pip install pre-commit, unpinned.
• .github/workflows/release-binaries.yml:223 and Makefile:92 — npx --yes @apidevtools/swagger-cli@^4, a caret range, not an exact version.
• .github/workflows/assign-reviews.yml:166-167 and todo-management.yml:156-157 — install openhands-sdk/openhands-tools straight from git+https://github.com/OpenHands/agent-sdk.git@main.
• .github/workflows/assign-reviews.yml:30 — downloads its agent driver script from raw.githubusercontent.com/OpenHands/agent-sdk/main/....
• .github/workflows/agent-server-rest-api-breakage.yml:36 — installs oasdiff via curl raw.githubusercontent.com/oasdiff/oasdiff/main/install.sh | version=1.19.1 sh. The tool version is pinned via env var, but the installer script itself is pulled from a mutable branch.
• .openhands/setup.sh:5,8 — floating curl astral.sh/uv/install.sh | sh, plus an explicit uv self update # always update to the latest version that force-upgrades an already-installed uv on every run.

Mutable GitHub Actions

• .github/workflows/qa-changes-by-openhands.yml:39 — OpenHands/extensions/plugins/qa-changes@main.
• 2 stray unpinned astral-sh/setup-uv@v7: persisted-settings-compat.yml:23, tests.yml:349 — the other ~24 call sites in this repo are already SHA-pinned; these two were just missed.
• 1 stray unpinned tj-actions/changed-files@v47: tests.yml:338 — every other occurrence is SHA-pinned.
• All official actions/* (checkout, setup-python, github-script, upload-artifact, download-artifact, setup-node) use moving major-version tags (@v6/@v7/@v8/@v9) throughout every workflow, never a SHA.

Pre-commit

• .pre-commit-config.yaml:4 — jumanjihouse/pre-commit-hook-yamlfmt pinned by tag (rev: 0.2.1), not a commit SHA.

Published wheel metadata (affects uvx/unlocked consumers, not normal repo CI — uv.lock covers in-repo dev/test)

• Completely unversioned: internal openhands-sdk edge (from openhands-tools and openhands-agent-server pyproject.toml), cachetools.
• Minimum-only (>=): agent-client-protocol, fastmcp, litellm, pydantic, browser-use, tree-sitter/tree-sitter-bash, httpx, fakeredis, libtmux, sqlalchemy, fastapi, uvicorn, websockets, alembic, aiosqlite, boto3, google-cloud-aiplatform, joserfc, deprecation, filelock, pillow, python-frontmatter, python-json-logger, tenacity, binaryornot, func-timeout, tom-swe, wsproto.
• Upper-bounded but still floats within range: lmnr (>=0.7.56,<0.8.0), toolshield (>=0.1.3,<0.2), docker (>=7.1,<8), openai (>=2.33.0,<3), posthog (>=6,<7).

Unpinned container/OS inputs

• Base images (tag, not digest): python:3.13-bookworm, nikolaik/python-nodejs:python3.13-nodejs22-slim, eclipse-temurin:17-jdk, golang:1.21-bookworm, ghcr.io/astral-sh/uv:0.11.6.
• 21 .github/workflows/*.yml occurrences of runs-on: ubuntu-latest (GitHub's rolling runner image can change preinstalled tool/library versions under an unchanged commit).
• Unversioned OS package installs via apt/apk/dnf/yum throughout CI.

Examples & docs (don't gate CI today, but get copy-pasted into user projects/workflows)

• 4 more unversioned mcp-server-fetch examples: 01_standalone_sdk/07_mcp_integration.py, 10_persistence.py, 13_get_llm_metrics.py, 05_skills_and_plugins/02_loading_plugins/example_plugins/code-quality/.mcp.json.
• 01_standalone_sdk/40_acp_agent_example.py — unversioned npx -y @agentclientprotocol/claude-agent-acp.
• examples/03_github_workflows/{01_basic_action,02_pr_review,03_todo_management,04_datadog_debugging} — install the SDK from agent-sdk.git@main / reference qa-changes@main in example workflow files users copy verbatim.

Suggested priority

1. Pin/remove the two active mcp-server-fetch invocations and the two Playwright auto-install call sites.
2. Fix the 3 stray unpinned Action refs (setup-uv × 2, tj-actions/changed-files × 1) to match the rest of the repo's SHA-pinning convention.
3. Replace @main git installs (assign-reviews/todo-management) and the raw.githubusercontent.com script fetches with pinned refs/SHAs.
4. Pin the pre-commit hook to a SHA and cap the wheel-metadata deps that currently have no upper bound at all.
5. Digest-pin the Docker base images; consider a checksum step for the ACP npm packages' transitive tree.