tensorflow / tensorflow/datasets

[ImageFolder] Calls to GCP

Open
#2,839 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
4.6k
Forks
1.6k
Avg merge
3h 54m
Merged PRs (30d)
1

Description

Short description
Using tfds.ImageFolder currently calls into GCP.

Environment information

  • Operating System: Ubuntu 18.04
  • Python version: 3.7
  • tensorflow-datasets/tfds-nightly version: nightly
  • tensorflow/tf-nightly version: nightly
  • Does the issue still exists with the last tfds-nightly package (pip install --upgrade tfds-nightly) ?
    Yes

Reproduction instructions

import tensorflow_datasets as tfds

print("Hello GCP")
builder = tfds.ImageFolder("data")

Link to logs

$ python3 test.py 
Hello GCP
2020-12-07 21:22:14.039783: W tensorflow/core/platform/cloud/google_auth_provider.cc:184] All attempts to get a Google authentication bearer token failed, returning an empty token. Retrieving token from files failed with "Not found: Could not locate the credentials file.". Retrieving token from GCE failed with "Failed precondition: Error executing an HTTP request: libcurl code 6 meaning 'Couldn't resolve host name', error details: Couldn't resolve host 'metadata'".

Expected behavior
Using ImageFolder does not send requests to GCP.

Additional context
Related to #2400

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reproduced entry point in test.py, focusing on tfds.ImageFolder("data") and the logged Google authentication attempt. Trace why this local ImageFolder construction reaches GCP, then verify that the same reproduction completes without GCP requests or authentication warnings.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, python, tensorflow
Domain
cloud, data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.