Google Colab - Instance Segmentation : Model couldn't start retraining (fine tuning), training quit immediately at INFO:tensorflow:Starting Queues. ^C

---

System information

• What is the top-level directory of the model you are using:
  /content/gdrive/My\ Drive/tensorflow_1.9/models/research
• Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
  Used modified pets_tf_record script to create tf_record. Script_file
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
  Using Google Colab
• TensorFlow installed from (source or binary):
  pip install --upgrade tensorflow==1.9
• TensorFlow version (use command below):
  both tensorflow repository and tensorflow library installed v1.9
• Bazel version (if compiling from source):
• CUDA/cuDNN version:
• GPU model and memory:
• Exact command to reproduce:

Describe the problem

I'm trying to retrain mask_rcnn_inception_v2 . As soon as I issue command !python object_detection/train.py --train_dir=object_detection/moved_data/training_inception --pipeline_config_path=object_detection/moved_data/training_inception/mask_rcnn_inception_v2_coco.config training halt and processing terminates. A note able thing is ram bar, it remains normal when I issue command, and loading of checkpoint etc keep logging, but just before termination, at first Ram bar shows 50% consumption for some seconds and then it turn to yellow and shows 80% consumption and program terminates.
Considering it some model bug, I tried to retrain mask_rcnn_resnet101 but it also stopped training after two more logging lines.

Source code / logs

After some below pasted warning and logging lines, inception was stopped at
INFO:tensorflow:Restoring parameters from /content/gdrive/My Drive/tensorflow_1.9/models/research/object_detection/moved_data/pre_trained_model/mask_inception/mask_rcnn_inception_v2_coco_2018_01_28/model.ckpt
INFO:tensorflow:Restoring parameters from /content/gdrive/My Drive/tensorflow_1.9/models/research/object_detection/moved_data/pre_trained_model/mask_inception/mask_rcnn_inception_v2_coco_2018_01_28/model.ckpt
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Starting Session.
INFO:tensorflow:Starting Session.
INFO:tensorflow:Saving checkpoint to path object_detection/moved_data/training_inception/model.ckpt
INFO:tensorflow:Saving checkpoint to path object_detection/moved_data/training_inception/model.ckpt
INFO:tensorflow:Starting Queues.
INFO:tensorflow:Starting Queues.
^C

and resnet101 executed two more lines and ended at
INFO:tensorflow:Restoring parameters from object_detection/moved_data/training_resnet/model.ckpt-0
INFO:tensorflow:Restoring parameters from object_detection/moved_data/training_resnet/model.ckpt-0
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Starting Session.
INFO:tensorflow:Starting Session.
INFO:tensorflow:Saving checkpoint to path object_detection/moved_data/training_resnet/model.ckpt
INFO:tensorflow:Saving checkpoint to path object_detection/moved_data/training_resnet/model.ckpt
INFO:tensorflow:Starting Queues.
INFO:tensorflow:Starting Queues.
INFO:tensorflow:global_step/sec: 0
INFO:tensorflow:global_step/sec: 0
INFO:tensorflow:Recording summary at step 0.
INFO:tensorflow:Recording summary at step 0.
^C
Both was terminated when ram bar turned to yellow, showing more than 80% usage

I went through many open and closed issues and solutions on on github, stackoverflow and other forums as well e.g. gitlab and some other website e.g. tensorflow original website documentation, everyone stated it could be a model's input problem, in other words it's a problem or error bug in tfrecords, train.record, test.record. So I observed and look closely on tfrecords file, and inspected them.
When I was converting data to tfrecord my directory structure was like
|--dataset (84 1280x720 pics)
--|--train
----|--Annotations
------|--masks dir
--------|--mask png files
------|--xmls dir
--------|--instance bound box xml files
----|--JPEGImages dir
------|--real images data
----|--label_map.pbtxt
--|--test (5 1280x720 pics)
----|--Annotations
------|--masks dir
--------|--mask png files
------|--xmls dir
--------|--instance bound box xml files
----|--JPEGImages dir
------|--real images data
----|--label_map.pbtxt
For inspection my code was,
import tensorflow as tf
for example in tf.python_io.tf_record_iterator("object_detection/moved_data/dataset/train/train.record-00000-of-00001"):
result = tf.train.Example.FromString(example)
print(result.features.ListFields())
break

and output was (first example image has 7 cotton plant in it, so mask, bound box and label, text etc. are also 7)
[(<google.protobuf.pyext._message.FieldDescriptor object at 0x7f3e7d1368d0>, {'image/key/sha256': bytes_list { value: "214bbd487a25966acddfa6c4209d8752ad8369f478b0fd40c453cee138b8bf31" } ,'image/object/mask': bytes_list { value: "\211PNG\r\n\032\n\000\000\000\rIHDR\000\000\00۔۔۔202\314(\341\000\000\000\000IEND\256B\202" value: "\211PNG\r\n\032\n\000\000\000\rIHDR\000\000\005۔۔۔\304\035\366\177\375\262\266\022\000\000\000\000IEND\256B\202" value: "\211PNG\r\n\032\n\000\000\000\rIHDR\000\000\005۔۔۔363?f8\312\3146\363\351\262\000\000\000\000IEND\256B\202"
۔ ۔ ۔
value: "\211PNG\r\n\032\n\000\000\000\rIHDR\000\000\005۔۔۔\000\341R\3727\022\001\204b\201\001\ }

, 'image/width': int64_list { value: 1280 }

, 'image/format': bytes_list { value: "jpeg" }

, 'image/object/class/label': int64_list { value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 }

, 'image/height': int64_list { value: 720 }

, 'image/object/class/text': bytes_list { value: "cotton" value: "cotton" value: "cotton" value: "cotton" value: "cotton" value: "cotton" value: "cotton" }

, 'image/source_id': bytes_list { value: "cotton0" }

, 'image/object/bbox/ymin': float_list { value: 0.32499998807907104 value: 0.3444444537162781 value: 0.644444465637207 value: 0.47083333134651184 value: 0.2916666567325592 value: 0.2708333432674408 value: 0.3263888955116272 }

'image/encoded': bytes_list { value: "\377\330\377\340\000\020JFIF\000\001\001\000\000\00...g\031\352\351R9\346\335\237\377\331}

, 'image/object/bbox/ymax': float_list { value: 0.47361111640930176 value: 0.5472221970558167 value: 0.762499988079071 value: 0.6666666865348816 value: 0.35555556416511536 value: 0.49861112236976624 value: 0.574999988079071 }

, 'image/object/bbox/xmin': float_list { value: 0.4320312440395355 value: 0.08671875298023224 value: 0.20703125 value: 0.37031251192092896 value: 0.907031238079071 value: 0.742968738079071 value: 0.63671875 }

, 'image/filename': bytes_list { value: "cotton0" }

, 'image/object/bbox/xmax': float_list { value: 0.5234375 value: 0.22499999403953552 value: 0.2914062440395355 value: 0.48359376192092896 value: 0.9429687261581421 value: 0.8890625238418579 value: 0.7640625238418579 } })]

this seems exactly right, at least as python script converte my real images, annotations xmls, masks.png to tf.record format expect it to be, I also decoded one of image using code provided by Tensorflow website tutorial for tf.data and it resulted as follow

But when I tried to parse tfrecord with

tf.parse_single_exapmle(tf_record, features = { 'image': tf.FixedLenFeature([], dtype=tf.string), 'label': tf.FixedLenFeature([], dtype=tf.int64) }
it throws an error
InvalidArgumentError: Feature: image (data type: string) is required but could not be found. [Op:ParseSingleExample]

and as my image variable name in tf_record structure is image/encoded instead of simple image, assuming this is reason, I changed
tf.parse_single_exapmle(tf_record, features = { 'image': tf.FixedLenFeature([], dtype=tf.string), 'label': tf.FixedLenFeature([], dtype=tf.int64) }
to
tf.parse_single_exapmle(tf_record, features = { 'image/encoded': tf.FixedLenFeature([], dtype=tf.string), 'image/object/class/label': tf.FixedLenFeature([], dtype=tf.int64) }
it gave new error
InvalidArgumentError: Key: image/object/class/label. Can't parse serialized Example. [[Node: ParseSingleExample/ParseSingleExample = ParseSingleExample[Tdense=[DT_STRING, DT_INT64], dense_keys=["image/encoded", "image/object/class/label"], dense_shapes=[[], []], num_sparse=0, sparse_keys=[], sparse_types=[]](arg0, ParseSingleExample/Const, ParseSingleExample/Const_1)]] [Op:IteratorGetNextSync]

I've searched alot on it, but didn't found my solution or this issue solved any where. Therefore I've posted it here as new issue. It is requested to please give guidelines to get out of it. Does model not starting training is it's input tf_record bug, or something else .
It seems new issue, my repository and tf itself version 1.9.