tensorflow / tensorflow/models

Is FasterRCNN still trainable ? Seems to produce NaN.

Open
#7,829 0 comments 0 reactions 3 assignees View on GitHub

@pkulzc is already working on this.

Since Jun 19, 2020.

models:research:odapi type:bug
Dominant language
Python
Stars
77.7k
Forks
44.8k
PR merge metrics
No merged PRs in 30d

Description

System information
  • What is the top-level directory of the model you are using:
    FasterRCNN

  • Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
    Used the Pet Detection and Running locally docs.

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04):18.04

  • TensorFlow installed from (source or binary):binary

  • TensorFlow version (use command below):1.14

  • Bazel version (if compiling from source):

  • CUDA/cuDNN version:9.0

  • GPU model and memory:V100

  • Exact command to reproduce:
    Run the Pet Detector model on a local system as per the documentation. Produces NaNs quickly.

You can collect some of this information using our environment capture script:

https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh

You can obtain the TensorFlow version with

python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"

Describe the problem

Describe the problem clearly here. Be sure to convey here why it's a bug in TensorFlow or a feature request.

Running the Pet Detector model on a local system as per the documentation. Produces NaNs quickly.

Source code / logs

Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.