tensorflow / tensorflow/models

Way too huge event files - Disabled feature for GPU in model_lib_v2.py

Open
#9,660 3 comments 0 reactions 3 assignees View on GitHub

@pkulzc is already working on this.

Since Jan 22, 2021.

models:research:odapi type:feature
Dominant language
Python
Stars
77.7k
Forks
44.8k
PR merge metrics
No merged PRs in 30d

Description

Prerequisites

Please answer the following question for yourself before submitting an issue.

  • I checked to make sure that this feature has not been requested already.

1. The entire URL of the file you are using

https://github.com/tensorflow/models/blob/master/research/object_detection/model_lib_v2.py#L553

2. Describe the feature you request

I would like training on gpu to write less often to event files, since it's a waste of I/O and disk space

3. Additional context

I wonder what is the reason for this feature to be disabled? Only a comment about GPU issues, while it seems to be working just fine on my setup.

if use_tpu:
  num_steps_per_iteration = 100
else:
  # TODO(b/135933080) Explore setting to 100 when GPU performance issues
  # are fixed.
  num_steps_per_iteration = 1

4. Are you willing to contribute it? (Yes or No)

Yes

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.