clab / clab/dynet

Could not find key /_1 in the model file even though the key is there

Open
#1,580 1 comment 0 reactions 0 assignees View on GitHub
major bug
Dominant language
C++
Stars
3.4k
Forks
701
PR merge metrics
No merged PRs in 30d

Description

To reproduce:
```
import dynet as dy

m = dy.ParameterCollection()
a = m.add_lookup_parameters((100, 100))
b = m.add_parameters(100)
t = dy.SimpleSGDTrainer(m)
e = dy.sum_elems(a)
e.forward()
(e/0).backward()
try:
t.update()
except RuntimeError as e:
print(e)
dy.save("/tmp/tmp.model", (a, b))

m2 = dy.ParameterCollection()
x = m2.add_lookup_parameters((100, 100))
y = m2.add_parameters(100)
dy.load("/tmp/tmp.model", m2)
```
I wrapped the `update()` in a try/except because I don't want the whole program to die just because one gradient is bad. It's definitely a problem that all updates fail because of bad gradients, but this error message did not help me realize this is the case.
Did I do something wrong, or is this a bug? It happened in my project using DyNet and it took a long time to figure out why (danielhers/tupa#79).

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by running the Python reproduction using ParameterCollection, SimpleSGDTrainer.update(), save(), and load(). Trace how the failed gradient update affects the later model-file lookup and identify where the reported error is produced. Done means the reproduced failure is handled or reported clearly enough to distinguish the bad-gradient cause from a missing key.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.