tensorflow / tensorflow/models

Are images from different TF-records contained in different folders shuffled to form the training batches?

Open
#10,353 0 comments 0 reactions 3 assignees View on GitHub

@pkulzc is already working on this.

Since Nov 9, 2021.

models:research:odapi type:support
Dominant language
Python
Stars
77.7k
Forks
44.8k
PR merge metrics
No merged PRs in 30d

Description

Hi,

I am trying to understand how the object_detection API deals with the training TF-records.
As you know, TF-records can be provided specifying as many input_path items as wanted in the train_input_reader section of the pipeline.config file.

For example:

  tf_record_input_reader {
    input_path: "asia/train_*.record"
    input_path: "america/train_*.record"
    input_path: "europe/train_*.record"
  }

My question is: in which order are the TF-records used in the training, are they shuffled? I understand the images inside each TF-record are shuffled, aren't they? But are the images of different TF-records from different folders included on a same batch?

I am afraid that on each batch there are only images of one continent (as it is how the folders containing my TF-records are organized, i.e. Asia, America, Europe...).

Best regards,
Rubén.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.