How to load a dataset from google cloud storage with tensorflow cloud?
Open
@pavithrasv is already working on this.
Since Aug 20, 2020.
bug
- Dominant language
- Python
- Stars
- 383
- Forks
- 93
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 1
Description
Tensorflow cloud configuration:
GCP_BUCKET = "stereo-train"
tfc.run(
requirements_txt="requirements.txt",
chief_config=tfc.MachineConfig(
cpu_cores=8,
memory=30,
accelerator_type=tfc.AcceleratorType.NVIDIA_TESLA_T4,
accelerator_count=1,
),
docker_image_bucket_name=GCP_BUCKET,
)
And I have a bucket called gs://stereo-train that contains the dataset. The exact location of the dataset is:
gs://stereo-train/data_scene_flow/training/dat
However, when using this location like so:
tf.keras.preprocessing.image_dataset_from_directory("gs://stereo-train/data_scene_flow/training/dat", image_size=(375,1242),\
batch_size=6, shuffle=False, label_mode=None)
Behavior:
getting the error that "gs://stereo-train/data_scene_flow/training/dat" doesn't exist
Expected behavior:
tf.keras.preprocessing.image_dataset_from_directory should know that there's a gs bucket associated with the account and the dataset should be loaded.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.