carpedm20 / carpedm20/NTM-tensorflow
Restore called with invalid save path
- Dominant language
- Jupyter Notebook
- Stars
- 1k
- Forks
- 212
- PR merge metrics
- No merged PRs in 30d
Description
When I run the test mode
$ python main.py --task copy --test_max_length 120 --max_length 130
it shows the error "Restore called with invalid save path" but the path exists! Anyone else encountered the same problem?
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so locally
{'checkpoint_dir': 'checkpoint',
'continue_train': None,
'controller_dim': 100,
'controller_layer_size': 1,
'epoch': 100000,
'input_dim': 10,
'is_train': False,
'max_length': 130,
'min_length': 1,
'output_dim': 10,
'read_head_size': 1,
'task': 'copy',
'test_max_length': 120,
'write_head_size': 1}
changepoint/copy_130
ln: failed to create symbolic link ‘copy_130/copy_10’: File exists
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:925] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
I tensorflow/core/common_runtime/gpu/gpu_device.cc:951] Found device 0 with properties:
name: GeForce GTX 960
major: 5 minor: 2 memoryClockRate (GHz) 1.253
pciBusID 0000:01:00.0
Total memory: 3.94GiB
Free memory: 229.88MiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:972] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] 0: Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 960, pci bus id: 0000:01:00.0)
[*] Building a NTM model
Percent: [####################] 100.00% Finished.
[*] Build a NTM model finished
[*] Reading checkpoints...
Traceback (most recent call last):
File "/home/user/Desktop/NTM-tensorflow-master/main.py", line 83, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 30, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "/home/user/Desktop/NTM-tensorflow-master/main.py", line 70, in main
ntm.load(FLAGS.checkpoint_dir, FLAGS.task)
File "/home/user/Desktop/NTM-tensorflow-master/ntm.py", line 266, in load
self.saver.restore(self.sess, os.path.join(checkpoint_dir, ckpt_name))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1342, in restore
"File path is: %r" % (save_path, file_path))
ValueError: Restore called with invalid save path: u'checkpoint/copy_130/NTM-copy_copy.model-3402'. File path is: u'checkpoint/copy_130/NTM-copy_copy.model-3402'
[Finished in 286.5s with exit code 1]
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the checkpoint-loading call in ntm.py line 266 and its caller in main.py line 70. Inspect the files under checkpoint/copy_130 and compare them with the checkpoint name being passed to restore. Done means test mode can load the existing checkpoint without the invalid save path error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100