felladrin / felladrin/MiniSearch

Replace the Hugging Face sync action with Trusted Publishers (OIDC)

Closed
#2,588 0 comments 0 reactions 0 assignees View on GitHub
enhancement security
Dominant language
TypeScript
Stars
587
Forks
70
Avg merge
1h 42m
Merged PRs (30d)
184

Description

### Problem

The `Deploy to Hugging Face` workflow pushes to the Space with a long-lived `HF_TOKEN` secret, through `JacobLinCool/huggingface-sync`, which we have to pin to a branch tip because its release tags point at commits without `dist/`. `renovate.json` carries a rule to leave that pin alone, for the same reason.

The Hub now supports [Trusted Publishers](https://huggingface.co/docs/hub/trusted-publishers): the job proves its identity with a GitHub OIDC token and gets back a Hugging Face token that is scoped to one repo and expires after an hour. Nothing to store, nothing to rotate, and no third-party action in the deploy path.

### Solution

Configure a trusted publisher on the Space, then replace the action step with the `hf` CLI. Everything the action does today has a replacement:

| Action input | Replacement |
|---|---|
| `user`, `space` | `$HF_SPACE`, still built from the `HF_SPACE_OWNER` and `HF_SPACE_NAME` variables |
| `token` | the OIDC exchange, nothing stored |
| `configuration` | a shell step that prepends the frontmatter to `README.md` |
| `github` | not needed (with `configuration` set, the action ignores the repo metadata it reads through that token, except for the URL it puts in the commit message) |

And the push itself becomes `hf upload "$HF_SPACE" . . --repo-type=space --delete="*"`, which uploads the whole checkout and deletes what disappeared, in one commit.

There is no explicit login step, and that is the point: on GitHub Actions the `hf` CLI [detects the provider, exchanges the OIDC token and uses the result](https://huggingface.co/docs/hub/trusted-publishers#quick-example-publish-a-model-from-github-actions), so the job only needs `permissions: id-token: write` and `HF_OIDC_RESOURCE` in its `env:` block (it is a job-level value, not a repository variable). The exchange lives in the base package (`huggingface_hub/_oidc.py`), no extra to install.

It fits this repo well: 283 tracked files, 2.2 MB, no LFS, and nothing tracked that the root `.gitignore` matches.

### Acceptance criteria

- `.github/workflows/deploy-to-hugging-face.yml` no longer references `JacobLinCool/huggingface-sync` or `secrets.HF_TOKEN`, and `renovate.json` no longer needs its `packageRules` key.
- A dispatch ends green and the Space reaches `RUNNING`. Check the Space page shows the new commit first, then `hf spaces wait Felladrin/MiniSearch --timeout 20m` exits 0. A green workflow only means the files were pushed: the build runs afterwards, and nothing in CI sees a `BUILD_ERROR`.
- The Space holds exactly the tree that was dispatched, deletions included. Run this from a checkout of the commit you dispatched (it prints nothing when they match, the Space has no `.gitattributes` of its own today, and it stays comparable as long as `git ls-files -i -c --exclude-standard` prints nothing):

```bash
diff <(git ls-files | LC_ALL=C sort) \
<(curl -s "https://huggingface.co/api/spaces/Felladrin/MiniSearch/tree/main?recursive=true" | jq -r '.[] | select(.type == "file") | .path' | LC_ALL=C sort)
```

- The Space `README.md` still starts with the frontmatter from `.github/hf-space-config.yml`, and the deployed app still gets the `custom_headers` (the in-browser AI needs the COOP/COEP pair, so a broken frontmatter shows up there first).
- The `HF_TOKEN` repository secret is deleted once a run from `main` has succeeded without it.

### Implementation notes

Size: **S**. One workflow file, one Renovate rule, the publisher on the Hub, one secret.

Steps 1, 2 and 6 happen in the Space settings and need the Write role on the Hub repo, so ping me for those.

1. On https://huggingface.co/spaces/Felladrin/MiniSearch/settings, open **Trusted Publishers** and add one with provider `GitHub Actions` and the claims `repository` = `felladrin/MiniSearch`, `branch` = `main`, `workflow` = `deploy-to-hugging-face.yml`. Claims are matched exactly, no wildcards.

2. Add a second, temporary entry with the same values except `branch` = the working branch, so the workflow can be tested before it reaches `main`. (A `workflow_dispatch` workflow that already exists on the default branch can be dispatched on any branch, and it runs that branch's version of the file.)

3. Rewrite `.github/workflows/deploy-to-hugging-face.yml`, and drop the whole `packageRules` key from `renovate.json` (the `JacobLinCool/huggingface-sync` rule is the only entry in it):

```yaml
name: Deploy to Hugging Face

on:
workflow_dispatch:

jobs:
sync-to-hf:
name: Sync to Hugging Face Spaces
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write # lets the job request the OIDC token
env:
HF_SPACE: ${{ vars.HF_SPACE_OWNER }}/${{ vars.HF_SPACE_NAME }}
HF_OIDC_RESOURCE: spaces/${{ vars.HF_SPACE_OWNER }}/${{ vars.HF_SPACE_NAME }}
steps:
- uses: actions/checkout@v7

- name: Install the Hugging Face CLI
run: |
pipx install huggingface_hub==1.31.0 # the OIDC exchange needs >= 1.19.0
hf version

- name: Prepend the Space configuration to README.md
run: |
{ echo '---'; cat .github/hf-space-config.yml; echo '---'; echo; cat README.md; } > README.space.md
mv README.space.md README.md

- name: Sync to the Space
run: |
hf upload "$HF_SPACE" . . \
--repo-type=space \
--delete="*" \
--commit-message="Sync from $GITHUB_REPOSITORY@${GITHUB_SHA:0:7}"
```

4. Dispatch the workflow from the branch and check the acceptance criteria. Note the Space then serves that branch, which is why step 5 re-deploys from `main`.

5. Merge, then dispatch from `main`. If nothing else landed in the meantime, `main` carries the tree the Space already has, so this uploads nothing and the Space does not rebuild. What it proves is the `branch` = `main` publisher entry: the upload has to authenticate before it can find nothing to do.

6. Delete the temporary publisher entry from step 2.

7. Delete the `HF_TOKEN` secret. `HF_SPACE_OWNER` and `HF_SPACE_NAME` stay as they are.

Worth knowing before you start:

- The root `.gitignore` becomes a deploy filter from now on: the CLI skips anything it matches, even when the file is tracked. Nothing is affected today, but a future tracked file under, say, `/client/dist` would silently not reach the Space.
- A dispatch that changes nothing no longer rebuilds the Space. The action force-pushes a fresh history on every run today, so every dispatch rebuilds; `hf upload` skips files that are already there and avoids empty commits. Add `hf spaces restart "$HF_SPACE"` as a step if we want the old behavior back.
- Space commits will be attributed to the synthetic `[OIDC]` user instead of `github-actions[bot]`, and the Space keeps its git history instead of getting a new one each run.
- `hf upload` always calls create-repo first, with `exist_ok=True` (see `cli/upload.py`), so the Hub answers 409 for a Space that exists and the CLI carries on. If the step instead fails there with a 403, the exchanged token does not cover that call, and the way around it is `HF_TOKEN="$(hf auth token)"` plus a two-line `HfApi().upload_folder(...)`, which uploads without creating anything.
- A failure with `invalid_grant` means the claims: check `branch` first (a dispatch from the wrong branch is the likely cause), then the `workflow` value. `invalid_request` means the `resource` itself is malformed, so check the `spaces/` prefix on `HF_OIDC_RESOURCE`. Both carry a Request ID for reporting upstream. A 401 means no exchange happened at all, so look at whether `HF_OIDC_RESOURCE` reached the job, and at the `id-token: write` permission.
- Renovate has no manager that reads a pip install inside a `run:` step, so the pin above will not be updated automatically. I'd rather bump it by hand than carry a `customManager` regex for a single line.

Installing the CLI with `curl -LsSf https://hf.co/cli/install.sh | bash`, as the Hub docs suggest, would also work. I prefer the pinned PyPI version: third-party steps in these workflows are pinned to a digest or a version, and an unpinned remote script is neither.

Contributor guide

Open the contributing guide

Research direction

Start with .github/workflows/deploy-to-hugging-face.yml, renovate.json, and .github/hf-space-config.yml, then configure the temporary Trusted Publisher entry before testing. Dispatch the workflow from the working branch and verify the Space reaches RUNNING, its tree and README frontmatter match the checkout, and no BUILD_ERROR occurs. After merging, dispatch from main and remove the temporary publisher and HF_TOKEN secret.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, python
Domain
ci-cd, cloud, devops, security
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.