huggingface / huggingface/datasets

Image Data loaded Twice

Open
#6,494 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
22k
Forks
3.4k
Avg merge
5d 7h
Merged PRs (30d)
17

Description

### Describe the bug

![1702472610561](https://github.com/huggingface/datasets/assets/28867010/4b7ef5e7-32c3-4b73-84cb-5de059caa0b6)
When I learn from https://huggingface.co/docs/datasets/image_load and try to load image data from a folder. I noticed that the image was read twice in the returned data. As you can see in the attached image, there are only four images in the train folder, but reading brings up eight images

### Steps to reproduce the bug

from datasets import Dataset, load_dataset

dataset = load_dataset("imagefolder", data_dir="data/", drop_labels=False)
# print(dataset["train"][0]["image"] == dataset["train"][1]["image"])

print(dataset)
print(dataset["train"]["image"])
print(len(dataset["train"]["image"]))

### Expected behavior

DatasetDict({
train: Dataset({
features: ['image', 'label'],
num_rows: 8
})
})
[, , , , , , , ]
8

### Environment info

- `datasets` version: 2.14.5
- Platform: Windows-10-10.0.22621-SP0
- Python version: 3.9.17
- Huggingface_hub version: 0.19.4
- PyArrow version: 13.0.0
- Pandas version: 2.0.3

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.