Invoking ptxas not supported on Windows

System information

• What is the top-level directory of the model you are using:
  \models\research\object_detection\

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
  NO, trying to use model_main.py

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
  Windows 10

• TensorFlow installed from (source or binary):
  Binary

• TensorFlow version (use command below):
  v1.15.0-rc2-10-g38ea9bbfea 1.15.0-rc3

• Bazel version (if compiling from source):
  N/A

• CUDA/cuDNN version:
  CUDA Version 10.0.130
  cuDNN: 7.6.4.38

• GPU model and memory:
  GeForce RTX 2080 SUPER. 8 GB dedicated, 32 GB shared

• Exact command to reproduce:
  From within an Anaconda environment:
  python model_main.py --alsologtostderr --model_dir=training/trial_1/ --pipeline_config_path=training/trial_1/faster_rcnn_nas_coco.config

Describe the problem

Hangs on a

W tensorflow/stream_executor/cuda/redzone_allocator.cc:312] Internal: Invoking ptxas not supported on Windows
Relying on driver to perform ptx compilation. This message will be only logged once.

message. Sits there forever. Sometimes (usually after restarting the terminal and clearing out any produced files like *.ckpt and *.pbtxt ) it gets passed this point and soon after crashes with an out of memory problem. Mentioning that because I don't know if its related or not.

Source code / logs

(tensorflow) F:\Hujber\TensorFlow\workspace\wormLearn>python model_main.py --alsologtostderr --model_dir=training/trial_1/ --pipeline_config_path=training/trial_1/faster_rcnn_nas_coco.config
2019-10-09 23:43:04.866391: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_100.dll
WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
  * https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\slim\nets\inception_resnet_v2.py:373: The name tf.GraphKeys is deprecated. Please use tf.compat.v1.GraphKeys instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\slim\nets\mobilenet\mobilenet.py:389: The name tf.nn.avg_pool is deprecated. Please use tf.nn.avg_pool2d instead.

WARNING:tensorflow:From model_main.py:109: The name tf.app.run is deprecated. Please use tf.compat.v1.app.run instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\utils\config_util.py:94: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

W1009 23:43:07.285009 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\utils\config_util.py:94: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\model_lib.py:573: The name tf.logging.warning is deprecated. Please use tf.compat.v1.logging.warning instead.

W1009 23:43:07.285009 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\model_lib.py:573: The name tf.logging.warning is deprecated. Please use tf.compat.v1.logging.warning instead.

WARNING:tensorflow:Forced number of epochs for all eval validations to be 1.
W1009 23:43:07.285009 15132 model_lib.py:574] Forced number of epochs for all eval validations to be 1.
WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\utils\config_util.py:480: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.

W1009 23:43:07.285009 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\utils\config_util.py:480: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.

INFO:tensorflow:Maybe overwriting train_steps: None
I1009 23:43:07.285009 15132 config_util.py:480] Maybe overwriting train_steps: None
INFO:tensorflow:Maybe overwriting sample_1_of_n_eval_examples: 1
I1009 23:43:07.285009 15132 config_util.py:480] Maybe overwriting sample_1_of_n_eval_examples: 1
INFO:tensorflow:Maybe overwriting eval_num_epochs: 1
I1009 23:43:07.300634 15132 config_util.py:480] Maybe overwriting eval_num_epochs: 1
INFO:tensorflow:Maybe overwriting load_pretrained: True
I1009 23:43:07.300634 15132 config_util.py:480] Maybe overwriting load_pretrained: True
INFO:tensorflow:Ignoring config override key: load_pretrained
I1009 23:43:07.300634 15132 config_util.py:490] Ignoring config override key: load_pretrained
WARNING:tensorflow:Expected number of evaluation epochs is 1, but instead encountered `eval_on_train_input_config.num_epochs` = 0. Overwriting `num_epochs` to 1.
W1009 23:43:07.316247 15132 model_lib.py:590] Expected number of evaluation epochs is 1, but instead encountered `eval_on_train_input_config.num_epochs` = 0. Overwriting `num_epochs` to 1.
INFO:tensorflow:create_estimator_and_inputs: use_tpu False, export_to_tpu False
I1009 23:43:07.316247 15132 model_lib.py:623] create_estimator_and_inputs: use_tpu False, export_to_tpu False
INFO:tensorflow:Using config: {'_model_dir': 'training/trial_1/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x00000260DABBD288>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
I1009 23:43:07.331873 15132 estimator.py:212] Using config: {'_model_dir': 'training/trial_1/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x00000260DABBD288>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
WARNING:tensorflow:Estimator's model_fn (<function create_model_fn.<locals>.model_fn at 0x00000260DABB5708>) includes params argument, but params are not passed to Estimator.
W1009 23:43:07.331873 15132 model_fn.py:630] Estimator's model_fn (<function create_model_fn.<locals>.model_fn at 0x00000260DABB5708>) includes params argument, but params are not passed to Estimator.
INFO:tensorflow:Not using Distribute Coordinator.
I1009 23:43:07.331873 15132 estimator_training.py:186] Not using Distribute Coordinator.
INFO:tensorflow:Running training and evaluation locally (non-distributed).
I1009 23:43:07.331873 15132 training.py:612] Running training and evaluation locally (non-distributed).
INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps None or save_checkpoints_secs 600.
I1009 23:43:07.347513 15132 training.py:700] Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps None or save_checkpoints_secs 600.
WARNING:tensorflow:From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\python\training\training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
W1009 23:43:07.363146 15132 deprecation.py:323] From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\python\training\training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\data_decoders\tf_example_decoder.py:167: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.

W1009 23:43:07.363146 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\data_decoders\tf_example_decoder.py:167: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\data_decoders\tf_example_decoder.py:182: The name tf.VarLenFeature is deprecated. Please use tf.io.VarLenFeature instead.

W1009 23:43:07.363146 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\data_decoders\tf_example_decoder.py:182: The name tf.VarLenFeature is deprecated. Please use tf.io.VarLenFeature instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\builders\dataset_builder.py:61: The name tf.gfile.Glob is deprecated. Please use tf.io.gfile.glob instead.

W1009 23:43:07.378762 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\builders\dataset_builder.py:61: The name tf.gfile.Glob is deprecated. Please use tf.io.gfile.glob instead.

WARNING:tensorflow:num_readers has been reduced to 1 to match input file shards.
W1009 23:43:07.378762 15132 dataset_builder.py:66] num_readers has been reduced to 1 to match input file shards.
WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\builders\dataset_builder.py:80: parallel_interleave (from tensorflow.contrib.data.python.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.experimental.parallel_interleave(...)`.
W1009 23:43:07.394386 15132 deprecation.py:323] From F:\Hujber\TensorFlow\models\research\object_detection\builders\dataset_builder.py:80: parallel_interleave (from tensorflow.contrib.data.python.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.experimental.parallel_interleave(...)`.
WARNING:tensorflow:From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\contrib\data\python\ops\interleave_ops.py:77: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.experimental.AUTOTUNE)` instead. If sloppy execution is desired, use `tf.data.Options.experimental_determinstic`.
W1009 23:43:07.394386 15132 deprecation.py:323] From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\contrib\data\python\ops\interleave_ops.py:77: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.experimental.AUTOTUNE)` instead. If sloppy execution is desired, use `tf.data.Options.experimental_determinstic`.
2019-10-09 23:43:07.875217: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library nvcuda.dll
2019-10-09 23:43:08.008609: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: GeForce RTX 2080 SUPER major: 7 minor: 5 memoryClockRate(GHz): 1.815
pciBusID: 0000:41:00.0
2019-10-09 23:43:08.015807: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_100.dll
2019-10-09 23:43:08.028156: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_100.dll
2019-10-09 23:43:08.034195: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cufft64_100.dll
2019-10-09 23:43:08.040674: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library curand64_100.dll
2019-10-09 23:43:08.047902: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusolver64_100.dll
2019-10-09 23:43:08.057279: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusparse64_100.dll
2019-10-09 23:43:08.069618: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll
2019-10-09 23:43:08.072360: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\anchor_generators\grid_anchor_generator.py:59: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
W1009 23:43:13.023561 15132 deprecation.py:323] From F:\Hujber\TensorFlow\models\research\object_detection\anchor_generators\grid_anchor_generator.py:59: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
WARNING:tensorflow:From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\python\autograph\converters\directives.py:119: The name tf.is_nan is deprecated. Please use tf.math.is_nan instead.

W1009 23:43:16.071676 15132 module_wrapper.py:139] From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\python\autograph\converters\directives.py:119: The name tf.is_nan is deprecated. Please use tf.math.is_nan instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\utils\ops.py:465: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
W1009 23:43:16.149807 15132 deprecation.py:323] From F:\Hujber\TensorFlow\models\research\object_detection\utils\ops.py:465: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\utils\ops.py:468: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W1009 23:43:16.149807 15132 deprecation.py:323] From F:\Hujber\TensorFlow\models\research\object_detection\utils\ops.py:468: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\python\autograph\converters\directives.py:119: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.

W1009 23:43:17.891674 15132 module_wrapper.py:139] From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\python\autograph\converters\directives.py:119: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.

WARNING:tensorflow:From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\python\autograph\converters\directives.py:119: The name tf.image.resize_images is deprecated. Please use tf.image.resize instead.

W1009 23:43:19.450865 15132 module_wrapper.py:139] From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\python\autograph\converters\directives.py:119: The name tf.image.resize_images is deprecated. Please use tf.image.resize instead.

WARNING:tensorflow:From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\python\autograph\converters\directives.py:119: The name tf.image.resize_nearest_neighbor is deprecated. Please use tf.compat.v1.image.resize_nearest_neighbor instead.

W1009 23:43:19.466491 15132 module_wrapper.py:139] From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\python\autograph\converters\directives.py:119: The name tf.image.resize_nearest_neighbor is deprecated. Please use tf.compat.v1.image.resize_nearest_neighbor instead.

WARNING:tensorflow:From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\python\autograph\converters\directives.py:119: The name tf.string_to_hash_bucket_fast is deprecated. Please use tf.strings.to_hash_bucket_fast instead.

W1009 23:43:21.049735 15132 module_wrapper.py:139] From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\python\autograph\converters\directives.py:119: The name tf.string_to_hash_bucket_fast is deprecated. Please use tf.strings.to_hash_bucket_fast instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\builders\dataset_builder.py:148: batch_and_drop_remainder (from tensorflow.contrib.data.python.ops.batching) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.batch(..., drop_remainder=True)`.
W1009 23:43:21.471656 15132 deprecation.py:323] From F:\Hujber\TensorFlow\models\research\object_detection\builders\dataset_builder.py:148: batch_and_drop_remainder (from tensorflow.contrib.data.python.ops.batching) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.batch(..., drop_remainder=True)`.
INFO:tensorflow:Calling model_fn.
I1009 23:43:21.487268 15132 estimator.py:1148] Calling model_fn.
WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\meta_architectures\faster_rcnn_meta_arch.py:162: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

W1009 23:43:21.502909 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\meta_architectures\faster_rcnn_meta_arch.py:162: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

2019-10-09 23:43:21.512084: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2019-10-09 23:43:21.529066: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: GeForce RTX 2080 SUPER major: 7 minor: 5 memoryClockRate(GHz): 1.815
pciBusID: 0000:41:00.0
2019-10-09 23:43:21.540071: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_100.dll
2019-10-09 23:43:21.544621: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_100.dll
2019-10-09 23:43:21.547872: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cufft64_100.dll
2019-10-09 23:43:21.553238: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library curand64_100.dll
2019-10-09 23:43:21.557124: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusolver64_100.dll
2019-10-09 23:43:21.565002: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusparse64_100.dll
2019-10-09 23:43:21.568175: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll
2019-10-09 23:43:21.572306: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2019-10-09 23:43:22.169167: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-10-09 23:43:22.172834: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0
2019-10-09 23:43:22.176589: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N
2019-10-09 23:43:22.182536: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/device:GPU:0 with 6269 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080 SUPER, pci bus id: 0000:41:00.0, compute capability: 7.5)
INFO:tensorflow:A GPU is available on the machine, consider using NCHW data format for increased speed on GPU.
I1009 23:43:22.178170 15132 nasnet.py:408] A GPU is available on the machine, consider using NCHW data format for increased speed on GPU.
WARNING:tensorflow:From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\contrib\layers\python\layers\layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
W1009 23:43:22.178170 15132 deprecation.py:323] From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\contrib\layers\python\layers\layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\slim\nets\nasnet\nasnet_utils.py:459: The name tf.train.get_or_create_global_step is deprecated. Please use tf.compat.v1.train.get_or_create_global_step instead.

W1009 23:43:22.287549 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\slim\nets\nasnet\nasnet_utils.py:459: The name tf.train.get_or_create_global_step is deprecated. Please use tf.compat.v1.train.get_or_create_global_step instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\core\anchor_generator.py:149: The name tf.assert_equal is deprecated. Please use tf.compat.v1.assert_equal instead.

W1009 23:43:29.248443 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\core\anchor_generator.py:149: The name tf.assert_equal is deprecated. Please use tf.compat.v1.assert_equal instead.

INFO:tensorflow:Scale of 0 disables regularizer.
I1009 23:43:29.248443 15132 regularizers.py:98] Scale of 0 disables regularizer.
WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\meta_architectures\faster_rcnn_meta_arch.py:986: The name tf.get_variable_scope is deprecated. Please use tf.compat.v1.get_variable_scope instead.

W1009 23:43:29.248443 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\meta_architectures\faster_rcnn_meta_arch.py:986: The name tf.get_variable_scope is deprecated. Please use tf.compat.v1.get_variable_scope instead.

INFO:tensorflow:Scale of 0 disables regularizer.
I1009 23:43:29.264083 15132 regularizers.py:98] Scale of 0 disables regularizer.
INFO:tensorflow:depth of additional conv before box predictor: 0
I1009 23:43:29.264083 15132 convolutional_box_predictor.py:148] depth of additional conv before box predictor: 0
WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\box_coders\faster_rcnn_box_coder.py:82: The name tf.log is deprecated. Please use tf.math.log instead.

W1009 23:43:29.560967 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\box_coders\faster_rcnn_box_coder.py:82: The name tf.log is deprecated. Please use tf.math.log instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\core\minibatch_sampler.py:81: The name tf.random_shuffle is deprecated. Please use tf.random.shuffle instead.

W1009 23:43:29.592233 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\core\minibatch_sampler.py:81: The name tf.random_shuffle is deprecated. Please use tf.random.shuffle instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\utils\ops.py:1085: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version.
Instructions for updating:
box_ind is deprecated, use box_indices instead
W1009 23:43:29.685991 15132 deprecation.py:506] From F:\Hujber\TensorFlow\models\research\object_detection\utils\ops.py:1085: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version.
Instructions for updating:
box_ind is deprecated, use box_indices instead
WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\meta_architectures\faster_rcnn_meta_arch.py:185: The name tf.AUTO_REUSE is deprecated. Please use tf.compat.v1.AUTO_REUSE instead.

W1009 23:43:29.701617 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\meta_architectures\faster_rcnn_meta_arch.py:185: The name tf.AUTO_REUSE is deprecated. Please use tf.compat.v1.AUTO_REUSE instead.

WARNING:tensorflow:From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\contrib\layers\python\layers\layers.py:1634: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
W1009 23:43:32.791057 15132 deprecation.py:323] From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\contrib\layers\python\layers\layers.py:1634: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
INFO:tensorflow:Scale of 0 disables regularizer.
I1009 23:43:32.806683 15132 regularizers.py:98] Scale of 0 disables regularizer.
INFO:tensorflow:Scale of 0 disables regularizer.
I1009 23:43:32.822310 15132 regularizers.py:98] Scale of 0 disables regularizer.
WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\meta_architectures\faster_rcnn_meta_arch.py:2235: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

W1009 23:43:32.837936 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\meta_architectures\faster_rcnn_meta_arch.py:2235: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\meta_architectures\faster_rcnn_meta_arch.py:2236: get_or_create_global_step (from tensorflow.contrib.framework.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.get_or_create_global_step
W1009 23:43:32.837936 15132 deprecation.py:323] From F:\Hujber\TensorFlow\models\research\object_detection\meta_architectures\faster_rcnn_meta_arch.py:2236: get_or_create_global_step (from tensorflow.contrib.framework.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.get_or_create_global_step
WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\utils\variables_helper.py:126: The name tf.train.NewCheckpointReader is deprecated. Please use tf.compat.v1.train.NewCheckpointReader instead.

W1009 23:43:32.853562 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\utils\variables_helper.py:126: The name tf.train.NewCheckpointReader is deprecated. Please use tf.compat.v1.train.NewCheckpointReader instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\model_lib.py:317: The name tf.train.init_from_checkpoint is deprecated. Please use tf.compat.v1.train.init_from_checkpoint instead.

W1009 23:43:32.869188 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\model_lib.py:317: The name tf.train.init_from_checkpoint is deprecated. Please use tf.compat.v1.train.init_from_checkpoint instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\core\losses.py:174: The name tf.losses.huber_loss is deprecated. Please use tf.compat.v1.losses.huber_loss instead.

W1009 23:43:35.818088 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\core\losses.py:174: The name tf.losses.huber_loss is deprecated. Please use tf.compat.v1.losses.huber_loss instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\core\losses.py:180: The name tf.losses.Reduction is deprecated. Please use tf.compat.v1.losses.Reduction instead.

W1009 23:43:35.818088 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\core\losses.py:180: The name tf.losses.Reduction is deprecated. Please use tf.compat.v1.losses.Reduction instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\core\losses.py:345: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.
Instructions for updating:

Future major versions of TensorFlow will allow gradients to flow
into the labels input on backprop by default.

See `tf.nn.softmax_cross_entropy_with_logits_v2`.

W1009 23:43:35.849340 15132 deprecation.py:323] From F:\Hujber\TensorFlow\models\research\object_detection\core\losses.py:345: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.
Instructions for updating:

Future major versions of TensorFlow will allow gradients to flow
into the labels input on backprop by default.

See `tf.nn.softmax_cross_entropy_with_logits_v2`.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\meta_architectures\faster_rcnn_meta_arch.py:2202: The name tf.get_collection is deprecated. Please use tf.compat.v1.get_collection instead.

W1009 23:43:35.989976 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\meta_architectures\faster_rcnn_meta_arch.py:2202: The name tf.get_collection is deprecated. Please use tf.compat.v1.get_collection instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\builders\optimizer_builder.py:52: The name tf.train.MomentumOptimizer is deprecated. Please use tf.compat.v1.train.MomentumOptimizer instead.

W1009 23:43:36.021228 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\builders\optimizer_builder.py:52: The name tf.train.MomentumOptimizer is deprecated. Please use tf.compat.v1.train.MomentumOptimizer instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\model_lib.py:359: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.

W1009 23:43:36.021228 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\model_lib.py:359: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\model_lib.py:369: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.

W1009 23:43:36.021228 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\model_lib.py:369: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\model_lib.py:472: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

W1009 23:43:48.484605 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\model_lib.py:472: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\model_lib.py:476: The name tf.add_to_collection is deprecated. Please use tf.compat.v1.add_to_collection instead.

W1009 23:43:49.816037 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\model_lib.py:476: The name tf.add_to_collection is deprecated. Please use tf.compat.v1.add_to_collection instead.

WARNING:tensorflow:From F:\Hujber\TensorFlow\models\research\object_detection\model_lib.py:477: The name tf.train.Scaffold is deprecated. Please use tf.compat.v1.train.Scaffold instead.

W1009 23:43:49.816037 15132 module_wrapper.py:139] From F:\Hujber\TensorFlow\models\research\object_detection\model_lib.py:477: The name tf.train.Scaffold is deprecated. Please use tf.compat.v1.train.Scaffold instead.

INFO:tensorflow:Done calling model_fn.
I1009 23:43:49.831650 15132 estimator.py:1150] Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
I1009 23:43:49.831650 15132 basic_session_run_hooks.py:541] Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
I1009 23:43:59.346498 15132 monitored_session.py:240] Graph was finalized.
2019-10-09 23:43:59.354769: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: GeForce RTX 2080 SUPER major: 7 minor: 5 memoryClockRate(GHz): 1.815
pciBusID: 0000:41:00.0
2019-10-09 23:43:59.368323: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_100.dll
2019-10-09 23:43:59.372370: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_100.dll
2019-10-09 23:43:59.376063: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cufft64_100.dll
2019-10-09 23:43:59.381228: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library curand64_100.dll
2019-10-09 23:43:59.384374: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusolver64_100.dll
2019-10-09 23:43:59.389889: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusparse64_100.dll
2019-10-09 23:43:59.393346: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll
2019-10-09 23:43:59.399778: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2019-10-09 23:43:59.404838: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-10-09 23:43:59.408820: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0
2019-10-09 23:43:59.413834: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N
2019-10-09 23:43:59.418801: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6269 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080 SUPER, pci bus id: 0000:41:00.0, compute capability: 7.5)
INFO:tensorflow:Restoring parameters from training/trial_1/model.ckpt-0
I1009 23:43:59.424629 15132 saver.py:1284] Restoring parameters from training/trial_1/model.ckpt-0
WARNING:tensorflow:From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\python\training\saver.py:1069: get_checkpoint_mtimes (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file utilities to get mtimes.
W1009 23:44:04.970287 15132 deprecation.py:323] From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\python\training\saver.py:1069: get_checkpoint_mtimes (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file utilities to get mtimes.
INFO:tensorflow:Running local_init_op.
I1009 23:44:07.828599 15132 session_manager.py:500] Running local_init_op.
INFO:tensorflow:Done running local_init_op.
I1009 23:44:09.031818 15132 session_manager.py:502] Done running local_init_op.
INFO:tensorflow:Saving checkpoints for 0 into training/trial_1/model.ckpt.
I1009 23:44:35.653282 15132 basic_session_run_hooks.py:606] Saving checkpoints for 0 into training/trial_1/model.ckpt.
2019-10-09 23:45:11.780278: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_100.dll
2019-10-09 23:45:14.790199: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll
2019-10-09 23:45:16.309576: W tensorflow/stream_executor/cuda/redzone_allocator.cc:312] Internal: Invoking ptxas not supported on Windows
Relying on driver to perform ptx compilation. This message will be only logged once.