huggingface / huggingface/diffusers

AI-generated image provenance metadata (EU AI Act deepfake disclosure)

Open
#13,359 6 comments 3 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

## Context

Diffusers generates images that get saved and distributed. Under EU AI Act Article 50 (August 2, 2026), AI-generated images disclosed publicly must carry transparency metadata. Deepfake disclosure becomes mandatory.

Currently, when you run:

```python
pipe = StableDiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0")
image = pipe("a sunset over mountains").images[0]
image.save("output.png")
```

The saved PNG has no metadata indicating it's AI-generated. A recipient has no way to know its origin.

## The Gap

Photos from cameras carry EXIF (camera model, GPS, timestamp). AI-generated images carry nothing. There's no standard EXIF/XMP field for:

- `AIGenerated: True`
- `AIModel: stable-diffusion-xl-base-1.0`
- `AIPrompt: "a sunset over mountains"` (optional)
- `GeneratedAt: 2026-03-28T12:00:00Z`

## Possible Approach

Diffusers could optionally embed provenance into saved images:

```python
image.save("output.png",
pnginfo=PngInfo() # already supported
)
# Could include AI provenance in PNG tEXt chunks or EXIF
```

This could be:
1. **Opt-in flag** — `pipe(..., embed_provenance=True)`
2. **Default on, opt-out** — more aggressive but helps compliance
3. **Separate utility** — `diffusers.utils.embed_provenance(image, model_name=...)`

## Why Diffusers Specifically

- Diffusers is the most popular image generation library
- Images are the primary target of EU AI Act deepfake provisions
- Pillow (which diffusers uses) already supports custom EXIF/XMP writing
- C2PA exists but requires Certificate Authority infrastructure — a lightweight EXIF-based approach would have much broader adoption

## Existing Formats

- [C2PA](https://c2pa.org/) — media provenance with PKI (heavy)
- [AKF](https://akf.dev) — lightweight JSON provenance for 20+ formats including images
- Custom EXIF/XMP — simplest, most compatible

Not advocating for any specific format — the important thing is that diffusers outputs carry *some* provenance. What does the team think?

Contributor guide

Open the contributing guide

Research direction

Start with the pipeline output and Pillow's image.save/PngInfo path shown in the issue; no repository file or test is named. Compare the proposed opt-in, default, and utility approaches with C2PA and custom metadata formats. Done requires an agreed provenance format and API behavior, plus coverage for saved PNG metadata.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.