tensorflow / tensorflow/models
deeplab - unable to train model_variant=nas_hnasnet
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.7k
- Forks
- 44.8k
- PR merge metrics
- No merged PRs in 30d
Description
Hello supporters
I am using tensorflow 1.13.1
with tensorflow model garden version 1.11
2. Describe the bug
I am trying to train for 100 epochs, export a frozen graph , execute it on a few images and check the runtime of each model variant.
i have managed to train with model_variants mobilenet_v2, resnet_v1_50, resnet_v1_50_Beta, resnet_v1_101, resnet_v1_101_beta, xception_41, xception_65, xception_71 , nas_pnasnet using my own dataset.
when i change the model_variant value to nas_hnasnet
train.py fails with the following error
Traceback (most recent call last):
File "/home/omerbrandis/ffrobotics/tensorflow-models-master-v1.11/research/deeplab/train.py", line 628, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "/home/omerbrandis/ffrobotics/tensorflow-models-master-v1.11/research/deeplab/train.py", line 524, in main
dataset.ignore_label)
File "/home/omerbrandis/ffrobotics/tensorflow-models-master-v1.11/research/deeplab/train.py", line 377, in _train_deeplab_model
reuse_variable=(i != 0))
File "/home/omerbrandis/ffrobotics/tensorflow-models-master-v1.11/research/deeplab/train.py", line 275, in _tower_loss
_build_deeplab(iterator, {common.OUTPUT_TYPE: num_of_classes}, ignore_label)
File "/home/omerbrandis/ffrobotics/tensorflow-models-master-v1.11/research/deeplab/train.py", line 235, in _build_deeplab
'total_training_steps': FLAGS.training_number_of_steps,
File "/home/omerbrandis/ffrobotics/tensorflow-models-master-v1.11/research/deeplab/model.py", line 313, in multi_scale_logits
nas_training_hyper_parameters=nas_training_hyper_parameters)
File "/home/omerbrandis/ffrobotics/tensorflow-models-master-v1.11/research/deeplab/model.py", line 553, in _get_logits
nas_training_hyper_parameters=nas_training_hyper_parameters)
File "/home/omerbrandis/ffrobotics/tensorflow-models-master-v1.11/research/deeplab/model.py", line 458, in extract_features
model_options.image_pooling_stride, padding='VALID')
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 182, in func_with_args
return func(*args, **current_args)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/layers/python/layers/layers.py", line 122, in avg_pool2d
outputs = layer.apply(inputs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/engine/base_layer.py", line 1227, in apply
return self.call(inputs, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/layers/base.py", line 530, in call
outputs = super(Layer, self).call(inputs, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/engine/base_layer.py", line 554, in call
outputs = self.call(inputs, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/layers/pooling.py", line 256, in call
data_format=conv_utils.convert_data_format(self.data_format, 4))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/nn_ops.py", line 2718, in avg_pool
name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_nn_ops.py", line 93, in avg_pool
data_format=data_format, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 3300, in create_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1823, in init
control_input_ops)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1662, in _create_c_op
raise ValueError(str(e))
ValueError: Negative dimension size caused by subtracting 33 from 17 for 'clone_0/AvgPool2D/AvgPool' (op: 'AvgPool') with input shapes: [?,17,17,200].
please advise,
Omer.
--
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.