tensorflow / tensorflow/models
build_imagenet_data.py requires validation images to be in synset directories
@marksandler2 is already working on this.
Since May 22, 2020.
- Dominant language
- Python
- Stars
- 77.7k
- Forks
- 44.8k
- PR merge metrics
- No merged PRs in 30d
Description
Update with requested information
Describe the problem
I want to prepare the ImageNet dataset for training InceptionV3 as described in research/slim/README.md. As I have already downloaded the ImageNet dataset for some previous project I wanted to reuse it by not downloading it again but rather just build the required data format for tensorflow. (On a side note, the download with the provided script does not work, as the URLs seem to be out of date.)
Now the problem is that the script research/slim/datasets/build_imagenet_data.py requires not only the training images but also the validation images to be in the synset directories like that:
data_dir/n01440764/ILSVRC2012_val_00000293.JPEG
data_dir/n01440764/ILSVRC2012_val_00000543.JPEG
But they don't come sorted into these directories when you log into image-net.org and download the validation dataset. The training images are sorted into sub-directories, and for those the script worked (I commented the processing of the validation images). The ground truth labels for the validation images come via the devkit download in a .txt file.
Did anyone else run into this problem? Any thoughts on how to address this apart from writing a script that sorts the validation images into folders based on the ground truth synset labels?
Thanks in advance!
System information
- What is the top-level directory of the model you are using:
models/research/slim - Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
- TensorFlow installed from (source or binary): source
- TensorFlow version (use command below): 1.14.0
- Bazel version (if compiling from source): 0.29.1
- Python version: 3.6.5
- CUDA/cuDNN version: N/A
- GPU model and memory: N/A
- Exact command to reproduce:
cd models/research/slim/datasets
python build_imagenet_data.py --train_directory=PATH_TO_IMAGE_NET/ILSVRC2012_img_train --validation_directory=PATH_TO_IMAGE_NET/ILSVRC2012_img_val --output_directory=PATH_TO_IMAGE_NET/image-net-out --imagenet_metadata_file=imagenet_metadata.txt --labels_file=imagenet_lsvrc_2015_synsets.txt --bounding_box_file=imagenet_2012_bounding_boxes.csv
The imagenet_2012_bounding_boxes.csv file can be created like this:
cd models/research/slim/datasets
python process_bounding_boxes.py PATH_TO_IMAGE_NET/ILSVRC2012_bbox_train_v2 imagenet_lsvrc_2015_synsets.txt | sort > imagenet_2012_bounding_boxes.csv
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.