tensorflow / tensorflow/datasets
Original kaggle diabetic_retinopathy_detection dataset corrupted?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 1.6k
- Avg merge
- 3h 54m
- Merged PRs (30d)
- 1
Description







What I need help with / What I was wondering
Since using tfds to load the diabetic retinopathy detection dataset requires you to manually download and unzip the files into folders, the first two pictures are the issue being raised when I was trying to unzipping the 2015 Kaggle dataset. It looks like the original zip file was corrupted. So tfds.load('diabetic_retinopathy_detection') was not successful to me.
In addition, I was also trying to test my models being trained on those images which were successfully unzipped. It looks like some images were truncated or not readable by my machine. The later pictures are my code and the error being raised in the testing process.
What I've tried so far
A good way to solve these problems is to find those resized images on Kaggle (e.g. the resized 2015 & 2019 blindness detection). The images can be successfully unzipped from there.
It would be nice if...
However, it would be nice if someone can make tfds diabetic_retinopathy_detection compatible to this dataset. It would save tons of time for loading the data.
Environment information
- Operating System: Debian 9 (on Google Cloud VM)
- Python version: 3.7.8
tensorflow-datasets/tfds-nightlyversion: tensorflow-dataset 3.2.1tensorflow/tensorflow-gpu/tf-nightly/tf-nightly-gpuversion: tensorflow 2.3.0/ tensorflow-gpu 2.3.1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing tfds.load('diabetic_retinopathy_detection') with the original 2015 Kaggle archive and inspect the reported unzip and truncated-image errors. Compare the archive and image behavior with the resized Kaggle data described in the issue. Done means the dataset loader can use the original dataset and the affected images load successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- data, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100