DetectionTeamUCAS / DetectionTeamUCAS/FPN_Tensorflow

is_training is not found when restore ckpt from mult_gpu_train

Open
#138 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
350
Forks
128
PR merge metrics
No merged PRs in 30d

Description

When I inference my data with the checkpoint obtained from the multi_gpu_train.py, it get an error as follow:

Traceback (most recent call last):
File "/root/anaconda3/envs/tensorflow12/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
return fn(*args)
File "/root/anaconda3/envs/tensorflow12/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/root/anaconda3/envs/tensorflow12/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.NotFoundError: Key **is_training** not found in checkpoint
[[{{node save/RestoreV2}} = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
[[{{node save/RestoreV2/_395}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_397_save/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

I can successfully inference the data with the checkpoints from train.py but it doesn't work when use the checkpoint from the multi_gpu_train.py because of the variable is_training. Please tell me why it happens. Thank you !

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.