huggingface / huggingface/diffusers

Opt-in weight integrity verification at load time (pinned SHA-256 hashes / manifest)

Open
#14,784 0 comments 0 reactions 0 assignees View on GitHub
feature-request pipelines
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

**Is your feature request related to a problem? Please describe.**

`from_pretrained` loads whatever weight files are on disk without any integrity verification at load time. Transport is content-addressed (LFS etag / Xet) and `hf cache verify` (huggingface_hub v0.32+) can re-check a cache directory against what the Hub currently serves, but both of those guarantees end before the actual load: a checkpoint that was tampered with after verification, a modified snapshot in a shared cache, or any local directory passed directly to `from_pretrained` is loaded as-is, and there is no way to pin *approved* weight hashes that get enforced when the model loads.

This matters more as automated pipelines and agents consume public checkpoints without a human in the loop. Recent work shows the threat is practical: backdoored latent world-model checkpoints can hijack downstream control while passing pre-deployment checks (arXiv 2609.15781, "When the World Lies", Sep 2026), and scanner-evading deserialization attacks on model files keep evolving (arXiv 2607.17503, "ShadowPickle", Jul 2026). Pinning a revision protects identity ("this is what was uploaded at that commit") but does not let a user detect that the bytes being loaded today differ from the ones they approved, and `hf cache verify` is a manual, cache-only step outside the load path.

**Describe the solution you'd like.**

An opt-in integrity check in the existing load path, e.g.:

```python
pipe = DiffusionPipeline.from_pretrained(
"org/model",
expected_hashes={"unet/diffusion_pytorch_model.safetensors": "", ...},
)
```

or equivalently `integrity_manifest="path/to/manifest.json"` (or a manifest discovered inside the local directory). Proposed behavior:

- Before weight materialization, hash each weight file with SHA-256 and compare against the pinned value.
- On mismatch, raise a clear dedicated error naming the file and both hashes; an optional flag could downgrade this to a warning for soft adoption.
- While the files are open anyway, surface tensor-shape mismatches between checkpoint and config as a clear early error instead of a deep load-time failure.
- Default behavior unchanged: no kwarg, no verification, no extra I/O.

Scope note: this verifies *your approved bytes are the bytes being loaded* (tampering, substitution, cache drift). It cannot certify that an upload you approved is benign — that would need external attestation and is out of scope.

**Describe the alternatives you've considered.**

- Revision pinning — necessary but not sufficient: no detection of post-download tampering, and no contract for local directories.
- `hf cache verify` — great post-hoc audit, but manual, cache-directory-only, checks against the Hub's *current* state rather than hashes the user pinned, and not wired into loading.
- Verification in huggingface_hub itself — plausible home, but users express trust at `from_pretrained`, so surfacing the contract in diffusers would match where the decision is actually made; happy to be told the Hub is the better layer.
- `safetensors` — removes arbitrary-code-execution risk on load, but says nothing about whether the weights are the ones the user expects.

**Additional context.**

Happy to work on this if maintainers think it fits. Concrete plan: a small additive change on the existing checkpoint-loading path plus regression tests — a known-clean checkpoint must load unchanged, and a tampered checkpoint (one flipped byte → SHA-256 mismatch) must raise the dedicated error; also report the false-positive rate on clean checkpoints.

Contributor guide

Open the contributing guide

Research direction

Start at DiffusionPipeline.from_pretrained and trace the existing checkpoint-loading path before weight materialization. Use the proposed regression cases: a known-clean checkpoint must load unchanged, while a checkpoint with one flipped byte must raise a dedicated error naming the file and both hashes; also assess the requested early tensor-shape mismatch reporting.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, security
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.