huggingface / huggingface/diffusers
LDM3D rgblike_to_depthmap truncates the 16-bit depth map back to the 8-bit input dtype
- Lingua principale
- Python
- Stelle
- 34.5k
- Fork
- 7.3k
- Merge medio
- 3g 3h
- PR unite (30g)
- 91
Descrizione
### Describe the bug
`VaeImageProcessorLDM3D.rgblike_to_depthmap` combines two 8-bit channels into a 16-bit depth value and then casts the result back to the **8-bit input dtype**, truncating it. `postprocess(output_type="pil")` then feeds that into `numpy_to_depth`, which builds a `mode="I;16"` image and fails.
### Reproduction
```python
import numpy as np
from diffusers.image_processor import VaeImageProcessorLDM3D
p = VaeImageProcessorLDM3D()
img = np.zeros((1, 8, 8, 6), dtype=np.uint8)
img[..., 1] = 200 # high byte
img[..., 2] = 100 # low byte
p.postprocess(img, output_type="pil")
```
### Logs
```
ValueError: buffer is not large enough
```
### Cause
`image_processor.py:1048-1075`. Both branches widen for the arithmetic and then narrow again:
```python
original_dtype = image.dtype
image_safe = image.to(torch.int32)
depth_map = image_safe[:, :, 1] * 256 + image_safe[:, :, 2]
return depth_map.to(original_dtype) # 16-bit result -> 8-bit dtype
```
The function's whole purpose is to produce a 16-bit depth map from 8-bit RGB, so casting back to the input dtype is never correct. The commented-out `# depth_map = depth_map.to(torch.uint16)` lines right there suggest this was known but left undone.
Introduced by #12546, which fixed a real NumPy 2.0 `uint8 * 256` overflow and replaced it with this truncation. `tests/others/test_image_processor.py` has no `VaeImageProcessorLDM3D` coverage, which is why it shipped.
### Proposed fix
Return the wider integer type instead of narrowing: numpy → `uint16` (matching the `mode="I;16"` consumer), torch → the `int32` combination.
### Coordination note
I should have opened this **before** the PR rather than after — the AI-agent guidelines ask for an explicit maintainer acknowledgment on an issue first, and I skipped that step. **#14200** already has the fix and a regression test; I'm filing this so the discussion has somewhere to live and so you can say "no, do it differently" without anyone having reviewed code first. Happy to close the PR and wait if you'd prefer to reset the order properly.
Separate, deliberately out of scope: `output_type="np"` passes float `[0, 1]` arrays into the same function, where the int cast collapses them to 0. That needs a decision about that branch's contract, so I left it alone rather than guessing.
### System Info
diffusers `main`, Python 3.12, numpy 2.x
### Who can help?
@yiyixuxu @sayakpaul
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Start with image_processor.py around lines 1048-1075 and reproduce the failure using the example in the issue. Then inspect tests/others/test_image_processor.py and the existing fix and regression test in PR #14200. Done means postprocess(output_type="pil") returns a valid 16-bit depth image without the buffer error.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- numpy, python, pytorch
- Ambito
- computer-vision
- Tipo di issue
- Bug
- Difficoltà
- 2/5
- Tempo stimato
- 1-3 ore
- Stato di attività
- Ferma
- Chiarezza
- Specificata chiaramente
- Idoneità per principianti
- 25/100