huggingface / huggingface/datasets
`Dataset.to_dict()` ignore `decode=True` with Image feature
- Dominant language
- Python
- Stars
- 22k
- Forks
- 3.4k
- Avg merge
- 5d 7h
- Merged PRs (30d)
- 17
Description
### Describe the bug
`Dataset.to_dict` seems to ignore the decoding instruction passed in features.
### Steps to reproduce the bug
```python
import datasets
import numpy as np
from PIL import Image
img = np.random.randint(0, 256, (5, 5, 3), dtype=np.uint8)
img = Image.fromarray(img)
features = datasets.Features({"image": datasets.Image(decode=True)})
dataset = datasets.Dataset.from_dict({"image": [img]}, features=features)
print({key: dataset[key] for key in dataset.column_names})
# {'image': []}
print(dataset.to_dict())
# {'image': [{'bytes': b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x05\x00\x00\x00\x05\x08\x02\x00\x00\x00\x02\r\xb1\xb2\x00\x00\x00[IDATx\x9c\x01P\x00\xaf\xff\x01\x13\x1b<7\xe7\xe0\xdc^6\xed\x04\xc7M\xd2\x9f\x00X\x1b\xb0?\x1ba\x15\xc5 o\xd0\x80\xbe\x19/\x01\xec\x95\x1f\x9f\xffj\xfa1\xa7\xc4X\xea\xbe\xa4g\x00\xc4\x15\xdeC\xc7 \xbbaqe\xc8\xb9\xa9q\xe7\x00,?M\xc0)\xdaD`}\xb1\xdci\x1e\xafC\xa9]%.@\xa6\xf0\xb3\x00\x00\x00\x00IEND\xaeB`\x82', 'path': None}]}
```
### Expected behavior
I would expect `{key: dataset[key] for key in dataset.column_names}` and `dataset.to_dict()` to be equivalent. If the previous behavior is expected, then it should be stated [in the doc](https://huggingface.co/docs/datasets/v2.14.4/en/package_reference/main_classes#datasets.Dataset.to_dict).
### Environment info
- `datasets` version: 2.14.4
- Platform: Linux-6.2.0-31-generic-x86_64-with-glibc2.35
- Python version: 3.9.12
- Huggingface_hub version: 0.15.1
- PyArrow version: 12.0.1
- Pandas version: 2.0.3
- Pillow 9.5.0
- numpy 1.25.2
Contributor guide
Assessment
This issue has not been assessed yet.