Comfy-Org / Comfy-Org/workflow_templates

manifest.json index sha256 hashes are stale on main; no CI gate catches template manifest drift

Open
#1,091 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
TypeScript
Stars
901
Forks
188
Avg merge
23h 10m
Merged PRs (30d)
86

Description

## Summary

`packages/core/src/comfyui_workflow_templates_core/manifest.json` on `main` records stale `sha256` values for `index.json` and all 12 locale variants. Running the documented sync step on a pristine checkout of `main` — with **no** edits of any kind — produces a 13-line diff.

## Reproduction

```bash
git clone https://github.com/Comfy-Org/workflow_templates
cd workflow_templates # currently at 5097de61
git status --porcelain # clean
python scripts/sync/sync_bundles.py
git diff --stat
# .../comfyui_workflow_templates_core/manifest.json | 26 +++++++++++-----------
```

The 13 entries whose hashes change:

`index.json`, `index.ar.json`, `index.es.json`, `index.fa.json`, `index.fr.json`, `index.ja.json`, `index.ko.json`, `index.mcp.json`, `index.pt-BR.json`, `index.ru.json`, `index.tr.json`, `index.zh.json`, `index.zh-TW.json`

## Cause

https://github.com/Comfy-Org/workflow_templates/pull/1088 ("Archive Getting Started category and Reve templates") moved templates from `templates/` into `archived/` and edited all 13 `templates/index.*.json` files. The manifest hash for each of those is computed over the **pip-filtered** index that `filter_index_for_pip` produces (`scripts/sync/sync_bundles.py:126-141`), so moving templates out changes every one of them. The manifest committed in that PR does not reflect the final state of those files.

`filter_index_for_pip` is a pure deterministic JSON transform with no environment dependence, so this is genuine drift and not a Python-version artifact — reproduced locally on 3.12.

## Why CI did not catch it

Neither manifest-related workflow actually compares committed state against generated state for templates:

- `.github/workflows/validate-manifests.yml` runs `sync_bundles.py` **first** and then validates, so the manifest it checks is always the freshly generated one. It only detects orphaned templates/blueprints, never staleness.
- `.github/workflows/version-check.yml` also re-runs the sync, but its commit step bails out unless the root `pyproject.toml` version changed: *"Root version unchanged; skipping commit (no version bump performed)"*. A PR that edits `templates/index.*.json` without triggering a version bump therefore leaves the manifest stale on `main`.

By contrast the blueprints path **is** gated — `.github/workflows/validate-blueprints.yml:65-73` re-runs `sync_blueprints.py` and `exit 1`s on any diff. Templates have no equivalent check.

Note that `README.md:170` states *"This step must be run before committing; CI will fail if the manifest/bundles are out of sync"*, which is not what the workflows currently do for templates.

## Impact

The published `comfyui-workflow-templates-core` package ships a manifest whose recorded `sha256` for all 13 index files does not match the index files actually packaged, so any consumer that verifies checksums against the manifest fails on them.

## Knock-on effect on unrelated PRs

Any PR that legitimately runs `sync_bundles.py` per the README picks up these 13 unrelated lines. https://github.com/Comfy-Org/workflow_templates/pull/1092 carries them for exactly this reason — they are not caused by that change.

## Suggested fix

1. Land a manifest-only refresh on `main` (`python scripts/sync/sync_bundles.py`, commit just the manifest).
2. Add a staleness gate for templates mirroring `validate-blueprints.yml:65-73` — re-run `sync_bundles.py` and fail on a non-empty diff — so this cannot recur silently. Alternatively, drop the version-bump condition from `version-check.yml`'s commit step so manifest changes are always committed back.
3. Either way, reconcile `README.md:170` with the actual behaviour.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with scripts/sync/sync_bundles.py and run it on a clean checkout to confirm the 13-line manifest diff. Compare .github/workflows/validate-manifests.yml with validate-blueprints.yml:65-73, then review version-check.yml and README.md:170. Done means the committed manifest matches generated hashes, template drift makes CI fail, and the README describes the actual gate.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, python
Domain
build-system, ci-cd
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.