huggingface / huggingface/diffusers
latent_consistency_models model/pipeline review
- Ngôn ngữ chính
- Python
- Star
- 34.5k
- Fork
- 7.3k
- Merge trung bình
- 3 ngày 3 giờ
- Pull request đã merge (30 ngày)
- 91
Mô tả
# `latent_consistency_models` model/pipeline review
Commit tested: `0f1abc4ae8b0eb2a3b40e82a310507281144c423`
Review performed against the repository review rules.
## Issue 1: Img2Img does not serialize `requires_safety_checker`
Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/latent_consistency_models/pipeline_latent_consistency_img2img.py#L216-L239
Problem:
`LatentConsistencyModelImg2ImgPipeline.__init__` accepts `requires_safety_checker`, but never calls `self.register_to_config(...)`. The text2img LCM pipeline does register it, so img2img saved configs lose this user choice.
Impact:
Pipelines constructed with `requires_safety_checker=False` do not persist that setting through config/save/load paths, causing inconsistent serialization behavior between the two LCM pipelines.
Reproduction:
```python
from diffusers import LatentConsistencyModelPipeline, LatentConsistencyModelImg2ImgPipeline
kwargs = dict(
vae=None,
text_encoder=None,
tokenizer=None,
unet=None,
scheduler=None,
safety_checker=None,
feature_extractor=None,
requires_safety_checker=False,
)
for cls in (LatentConsistencyModelPipeline, LatentConsistencyModelImg2ImgPipeline):
pipe = cls(**kwargs)
print(cls.__name__, pipe.config.get("requires_safety_checker"))
# Text2Img prints False; Img2Img prints None.
```
Relevant precedent:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/latent_consistency_models/pipeline_latent_consistency_text2img.py#L211-L223
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_img2img.py#L314-L322
Suggested fix:
```python
self.vae_scale_factor = 2 ** (len(self.vae.config.block_out_channels) - 1) if getattr(self, "vae", None) else 8
self.image_processor = VaeImageProcessor(vae_scale_factor=self.vae_scale_factor)
self.register_to_config(requires_safety_checker=requires_safety_checker)
```
Duplicate search:
No matching issue or PR found for `LatentConsistencyModelImg2ImgPipeline requires_safety_checker`.
## Issue 2: Img2Img accepts a safety checker without a feature extractor
Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/latent_consistency_models/pipeline_latent_consistency_img2img.py#L216-L239
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/latent_consistency_models/pipeline_latent_consistency_img2img.py#L494-L506
Problem:
The img2img constructor does not reject `safety_checker != None` with `feature_extractor=None`. Later, `run_safety_checker` unconditionally calls `self.feature_extractor(...)`, producing a late `TypeError`.
Impact:
Users can construct an invalid pipeline successfully and only fail during inference/safety checking with an unclear `'NoneType' object is not callable` error.
Reproduction:
```python
import torch
from diffusers import LatentConsistencyModelImg2ImgPipeline
class DummySafetyChecker:
def __call__(self, images, clip_input):
return images, [False] * images.shape[0]
pipe = LatentConsistencyModelImg2ImgPipeline(
vae=None,
text_encoder=None,
tokenizer=None,
unet=None,
scheduler=None,
safety_checker=DummySafetyChecker(),
feature_extractor=None,
requires_safety_checker=True,
)
try:
pipe.run_safety_checker(torch.zeros(1, 3, 8, 8), "cpu", torch.float32)
except Exception as e:
print(type(e).__name__, str(e))
# TypeError 'NoneType' object is not callable
```
Relevant precedent:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/latent_consistency_models/pipeline_latent_consistency_text2img.py#L205-L209
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_img2img.py#L279-L283
Suggested fix:
```python
if safety_checker is not None and feature_extractor is None:
raise ValueError(
"Make sure to define a feature extractor when loading {self.__class__} if you want to use the safety"
" checker. If you do not want to use the safety checker, you can pass `'safety_checker=None'` instead."
)
```
Duplicate search:
No matching issue or PR found for `LatentConsistencyModelImg2ImgPipeline feature_extractor`.
## Issue 3: LCM pipeline `__call__` docs have stale parameters and defaults
Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/latent_consistency_models/pipeline_latent_consistency_text2img.py#L642-L690
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/latent_consistency_models/pipeline_latent_consistency_img2img.py#L711-L760
Problem:
Both `__call__` docstrings say `num_inference_steps` defaults to `50` and `guidance_scale` defaults to `7.5`, while the signatures default to `4` and `8.5`. The img2img docstring also documents nonexistent `height` and `width` parameters and omits its real `image` and `strength` parameters.
Impact:
The generated API docs mislead users about LCM's fast-step defaults and img2img inputs.
Reproduction:
```python
import inspect
from diffusers import LatentConsistencyModelPipeline, LatentConsistencyModelImg2ImgPipeline
for cls in (LatentConsistencyModelPipeline, LatentConsistencyModelImg2ImgPipeline):
sig = inspect.signature(cls.__call__)
doc = inspect.getdoc(cls.__call__) or ""
print(cls.__name__, sig.parameters["num_inference_steps"].default, sig.parameters["guidance_scale"].default)
print("doc says steps default 50:", "defaults to 50" in doc)
print("doc says guidance default 7.5:", "defaults to 7.5" in doc)
img_doc = inspect.getdoc(LatentConsistencyModelImg2ImgPipeline.__call__) or ""
print("img2img signature has image:", "image" in inspect.signature(LatentConsistencyModelImg2ImgPipeline.__call__).parameters)
print("img2img docs image:", "image (`" in img_doc)
print("img2img docs height:", "height (`int`" in img_doc)
```
Relevant precedent:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_img2img.py#L891-L902
Suggested fix:
Update the LCM docstrings to match their signatures: `num_inference_steps` default `4`, `guidance_scale` default `8.5`, and for img2img replace the stale `height`/`width` entries with `image` and `strength` documentation.
Duplicate search:
No matching issue or PR found for LCM img2img docstring/default mismatches.
Coverage and duplicate-search status:
Fast and slow tests exist for both LCM pipelines under `tests/pipelines/latent_consistency_models/`; slow coverage is not missing. Local target pytest collection was blocked by the venv torch build missing `torch._C._distributed_c10d` via shared test utilities. `python utils/check_copies.py` passed. Broad duplicate searches found historical LCM issues/PRs, but no duplicates for the three findings above.
Hướng dẫn đóng góp
Hướng nghiên cứu
Bắt đầu với src/diffusers/pipelines/latent_consistency_models/pipeline_latent_consistency_img2img.py và pipeline_latent_consistency_text2img.py, so sánh các constructor được nêu, cách xử lý safety-checker và các docstring của __call__. Chạy các test nhanh và chậm trong tests/pipelines/latent_consistency_models/ cùng với python utils/check_copies.py; được xem là hoàn thành khi đã bao phủ việc duy trì cấu hình, các tổ hợp safety-checker không hợp lệ và các tham số cùng giá trị mặc định được tài liệu hóa của cả hai pipeline.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- python, pytorch
- Lĩnh vực
- documentation, machine-learning, testing-qa
- Loại issue
- Lỗi
- Độ khó
- 3/5
- Thời gian dự kiến
- 1-2 ngày
- Mức độ hoạt động
- Ít trao đổi
- Độ rõ ràng
- Đặc tả rõ ràng
- Mức phù hợp với người mới
- 64/100