tensorflow / tensorflow/datasets

How to access to already downloaded ImageNet dataset?

Open
#2,748 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

help
Dominant language
Python
Stars
4.6k
Forks
1.6k
Avg merge
3h 54m
Merged PRs (30d)
1

Description

I downloaded two folders : ILSVRC2012_img_train.tar and ILSVRC2012_img_val.tar, and placed them in a drive with the following path: "H:\lizeth\imagenet2012" I additionally extracted both folders in the same directory (keeping the same directory organisation). As for the ILSVRC2012_img_train.tar after extraction, it contains 1001 .tar files too, which I extracted and keep them in the extracted ILSVRC2012_img_train folder.

I am using the following lines to get access to the data (validation only):

def load_ImageNet_test(ds_type, ds_dir):
    ds_test, ds_info = tfds.load(ds_type, split='validation',
                                             data_dir=ds_dir, download=False, shuffle_files=True,
                                             as_supervised=True, with_info=True)
    return ds_test, ds_info

BASEDIR = "H:\lizeth\imagenet2012"
ds_test, ds_info = load_ImageNet_test('imagenet2012', BASEDIR)

However I keep having same Assertion Error:

AssertionError: Dataset imagenet2012: could not find data in H:\lizeth\imagenet2012. Please make sure to call dataset_builder.download_and_prepare(), or pass download=True to tfds.load() before trying to access the tf.data.Dataset object.

If I set download = True I have the following error:

AssertionError: ImageNet requires manual download of the data. Please download the train and val set and place them into: H:\lizeth\imagenet2012\downloads\manual\imagenet2012\ILSVRC2012_img_train.tar, H:\lizeth\imagenet2012\downloads\manual\imagenet2012\ILSVRC2012_img_val.tar

This error is happening even though I moved the datasets files (zip and unzipped) there .. I'd really appreciate your help with this issue ...

Could you please provide the steps in detail on what is the correct procedure to access to the downloaded ImageNet dataset. Thanks !

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review the reported tfds.load('imagenet2012', ...) call and the manual ImageNet paths in the issue. Document the expected placement and preparation of the train and validation archives, and verify that the validation-only load succeeds with download disabled.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
data, machine-learning
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.