[deeplab] Eval.py not showing any result

System information

• What is the top-level directory of the model you are using: deeplab
• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): N/A
• TensorFlow installed from (source or binary):source
• TensorFlow version (use command below):1.10.0
• Exact command to reproduce: eval.py
• **Bazel version: N/A
• **CUDA/cuDNN version: N/A
• **GPU model and memory: N/A

Describe the problem

Hello, my problem is that when trying to run eval.py it never shows any kind of result. Log says it starts evaluating but there is not any kind of feedback about what miou it has. I've tried to run local_test.sh and the mobilenet_v2 version and it happens with both scripts.

And also when I visualize the images using vis.py it seems the model hasn't trained that well because all the images are black with small color traces.

Source code / logs

INFO:tensorflow:Waiting for new checkpoint at /root/workspace/TFG_Code/TFG_Seman tic_segmentation/src/CNN/DeepLab/research/deeplab/datasets/pascal_voc_seg/exp/tr ain_on_trainval_set/train
INFO:tensorflow:Found new checkpoint at /root/workspace/TFG_Code/TFG_Semantic_se gmentation/src/CNN/DeepLab/research/deeplab/datasets/pascal_voc_seg/exp/train_on trainval_set/train/model.ckpt-10
INFO:tensorflow:Graph was finalized.
2019-04-12 12:39:07.213731: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA
2019-04-12 12:39:09.598712: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1 405] Found device 0 with properties:
name: TITAN Xp major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:1c:00.0
totalMemory: 11.91GiB freeMemory: 11.76GiB
2019-04-12 12:39:09.741244: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1 405] Found device 1 with properties:
name: TITAN Xp major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:3f:00.0
totalMemory: 11.91GiB freeMemory: 11.76GiB
2019-04-12 12:39:09.884623: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1 405] Found device 2 with properties:
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:40:00.0
totalMemory: 10.92GiB freeMemory: 10.77GiB
2019-04-12 12:39:10.046895: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1 405] Found device 3 with properties:
name: Quadro P5000 major: 6 minor: 1 memoryClockRate(GHz): 1.7335
pciBusID: 0000:1d:00.0
totalMemory: 15.90GiB freeMemory: 15.79GiB
2019-04-12 12:39:10.051374: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1 484] Adding visible gpu devices: 0, 1, 2, 3
2019-04-12 12:39:12.112522: I tensorflow/core/common_runtime/gpu/gpu_device.cc:9 65] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-04-12 12:39:12.112567: I tensorflow/core/common_runtime/gpu/gpu_device.cc:9 71] 0 1 2 3
2019-04-12 12:39:12.112577: I tensorflow/core/common_runtime/gpu/gpu_device.cc:9 84] 0: N Y Y N
2019-04-12 12:39:12.112600: I tensorflow/core/common_runtime/gpu/gpu_device.cc:9 84] 1: Y N Y N
2019-04-12 12:39:12.112608: I tensorflow/core/common_runtime/gpu/gpu_device.cc:9 84] 2: Y Y N N
2019-04-12 12:39:12.112615: I tensorflow/core/common_runtime/gpu/gpu_device.cc:9 84] 3: N N N N
2019-04-12 12:39:12.114313: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1 097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 wit h 11378 MB memory) -> physical GPU (device: 0, name: TITAN Xp, pci bus id: 0000: 1c:00.0, compute capability: 6.1)
2019-04-12 12:39:12.269795: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1 097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:1 wit h 11378 MB memory) -> physical GPU (device: 1, name: TITAN Xp, pci bus id: 0000: 3f:00.0, compute capability: 6.1)
2019-04-12 12:39:12.422934: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1 097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:2 wit h 10421 MB memory) -> physical GPU (device: 2, name: GeForce GTX 1080 Ti, pci bu s id: 0000:40:00.0, compute capability: 6.1)
2019-04-12 12:39:12.582819: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1 097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:3 wit h 15296 MB memory) -> physical GPU (device: 3, name: Quadro P5000, pci bus id: 0 000:1d:00.0, compute capability: 6.1)
INFO:tensorflow:Restoring parameters from /root/workspace/TFG_Code/TFG_Semantic segmentation/src/CNN/DeepLab/research/deeplab/datasets/pascal_voc_seg/exp/train_ on_trainval_set/train/model.ckpt-10
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Starting evaluation at 2019-04-12-12:39:15
INFO:tensorflow:Visualizing on val set
INFO:tensorflow:Performing single-scale test.
INFO:tensorflow:Waiting for new checkpoint at /root/workspace/TFG_Code/TFG_Semantic_segmentation/src/CNN/DeepLab/research/deeplab/datasets/pascal_voc_seg/exp/train_on_trainval_set/train
INFO:tensorflow:Found new checkpoint at /root/workspace/TFG_Code/TFG_Semantic_segmentation/src/CNN/DeepLab/research/deeplab/datasets/pascal_voc_seg/exp/train_on_trainval_set/train/model.ckpt-10
INFO:tensorflow:Starting visualization at 2019-04-12-12:40:31
INFO:tensorflow:Visualizing with model /root/workspace/TFG_Code/TFG_Semantic_segmentation/src/CNN/DeepLab/research/deeplab/datasets/pascal_voc_seg/exp/train_on_trainval_set/train/model.ckpt-10
INFO:tensorflow:Graph was finalized.
2019-04-12 12:40:33.275125: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA

And also a long message at the beggining regarding incomplete shapes on ops:

4 ops no flops stats due to incomplete shapes.
Parsing Inputs...

=========================Options=============================
-max_depth 10000
-min_bytes 0
-min_peak_bytes 0
-min_residual_bytes 0
-min_output_bytes 0
-min_micros 0
-min_accelerator_micros 0
-min_cpu_micros 0
-min_params 0
-min_float_ops 1
-min_occurrence 0
-step -1
-order_by float_ops
-account_type_regexes .*
-start_name_regexes .*
-trim_name_regexes
-show_name_regexes .*
-hide_name_regexes
-account_displayed_op_only true
-select float_ops
-output stdout:

I've looked for everywhere but i can not find anything regarding why this is happening. Thank you!