Comfy-Org / Comfy-Org/workflow_templates
manifest.json index sha256 hashes are stale on main; no CI gate catches template manifest drift
- Dominant language
- TypeScript
- Stars
- 901
- Forks
- 188
- Avg merge
- 23h 10m
- Merged PRs (30d)
- 86
Description
## Summary
`packages/core/src/comfyui_workflow_templates_core/manifest.json` on `main` records stale `sha256` values for `index.json` and all 12 locale variants. Running the documented sync step on a pristine checkout of `main` — with **no** edits of any kind — produces a 13-line diff.
## Reproduction
```bash
git clone https://github.com/Comfy-Org/workflow_templates
cd workflow_templates # currently at 5097de61
git status --porcelain # clean
python scripts/sync/sync_bundles.py
git diff --stat
# .../comfyui_workflow_templates_core/manifest.json | 26 +++++++++++-----------
```
The 13 entries whose hashes change:
`index.json`, `index.ar.json`, `index.es.json`, `index.fa.json`, `index.fr.json`, `index.ja.json`, `index.ko.json`, `index.mcp.json`, `index.pt-BR.json`, `index.ru.json`, `index.tr.json`, `index.zh.json`, `index.zh-TW.json`
## Cause
https://github.com/Comfy-Org/workflow_templates/pull/1088 ("Archive Getting Started category and Reve templates") moved templates from `templates/` into `archived/` and edited all 13 `templates/index.*.json` files. The manifest hash for each of those is computed over the **pip-filtered** index that `filter_index_for_pip` produces (`scripts/sync/sync_bundles.py:126-141`), so moving templates out changes every one of them. The manifest committed in that PR does not reflect the final state of those files.
`filter_index_for_pip` is a pure deterministic JSON transform with no environment dependence, so this is genuine drift and not a Python-version artifact — reproduced locally on 3.12.
## Why CI did not catch it
Neither manifest-related workflow actually compares committed state against generated state for templates:
- `.github/workflows/validate-manifests.yml` runs `sync_bundles.py` **first** and then validates, so the manifest it checks is always the freshly generated one. It only detects orphaned templates/blueprints, never staleness.
- `.github/workflows/version-check.yml` also re-runs the sync, but its commit step bails out unless the root `pyproject.toml` version changed: *"Root version unchanged; skipping commit (no version bump performed)"*. A PR that edits `templates/index.*.json` without triggering a version bump therefore leaves the manifest stale on `main`.
By contrast the blueprints path **is** gated — `.github/workflows/validate-blueprints.yml:65-73` re-runs `sync_blueprints.py` and `exit 1`s on any diff. Templates have no equivalent check.
Note that `README.md:170` states *"This step must be run before committing; CI will fail if the manifest/bundles are out of sync"*, which is not what the workflows currently do for templates.
## Impact
The published `comfyui-workflow-templates-core` package ships a manifest whose recorded `sha256` for all 13 index files does not match the index files actually packaged, so any consumer that verifies checksums against the manifest fails on them.
## Knock-on effect on unrelated PRs
Any PR that legitimately runs `sync_bundles.py` per the README picks up these 13 unrelated lines. https://github.com/Comfy-Org/workflow_templates/pull/1092 carries them for exactly this reason — they are not caused by that change.
## Suggested fix
1. Land a manifest-only refresh on `main` (`python scripts/sync/sync_bundles.py`, commit just the manifest).
2. Add a staleness gate for templates mirroring `validate-blueprints.yml:65-73` — re-run `sync_bundles.py` and fail on a non-empty diff — so this cannot recur silently. Alternatively, drop the version-bump condition from `version-check.yml`'s commit step so manifest changes are always committed back.
3. Either way, reconcile `README.md:170` with the actual behaviour.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with scripts/sync/sync_bundles.py and run it on a clean checkout to confirm the 13-line manifest diff. Compare .github/workflows/validate-manifests.yml with validate-blueprints.yml:65-73, then review version-check.yml and README.md:170. Done means the committed manifest matches generated hashes, template drift makes CI fail, and the README describes the actual gate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, python
- Domain
- build-system, ci-cd
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100