Resource exhausted by faster rcnn and ssd restnet

i'm using model_main.py for training.

The batch size was already 1 and i've change the fixed_shape_resizer as 500x500 (using faster rcnn models) and

  session_config = tf.ConfigProto()
  session_config.gpu_options.per_process_gpu_memory_fraction = 0.3
  config = tf.estimator.RunConfig(model_dir=FLAGS.model_dir, session_config=session_config)

is also set.

But it keep showing (by ssd resnet, same error) :

2019-07-08 18:37:10.194834: W tensorflow/core/framework/op_kernel.cc:1502] OP_REQUIRES failed at cwise_ops_common.cc:70 : Resource exhausted: OOM when allocating tensor with shape[100,51150] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
until it break automatically.
I can only train the ssd mobilenet. very confusing.

i'm using gtx 1060 6G and RTX 2070. both has same error

Originally posted by @heizie in https://github.com/tensorflow/models/issues/1993#issuecomment-509300382