clab / clab/dynet

Trying to reuse LSTM weights in Keras from a model built in DyNet

Open
#1,646 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
3.4k
Forks
701
PR merge metrics
No merged PRs in 30d

Description

Hi :wave:

I am currently trying to use weights from a model in dynet:

dynet model:
```
dy_model = dy.BiRNNBuilder(layers, dim_input, dim_hidden, model, dy.LSTMBuilder)
```

Keras model:
```
def multi_layer_birnn(input, layer_count, dim_hidden, params):
out = input
dim_hidden = int(dim_hidden / 2)
for li in range(layer_count):
layer_params = params[li]
fwparams = layer_params['fw']
bwparams = layer_params['bw']
forward_cell = layers.LSTM(
dim_hidden,
return_sequences=True,
unit_forget_bias=False,
kernel_initializer =keras.initializers.Constant(fwparams['W']),
recurrent_initializer=keras.initializers.Constant(fwparams['U']),
bias_initializer=keras.initializers.Constant(fwparams['b']),
)
forward_cell.trainable = False

backward_cell = layers.LSTM(
dim_hidden,
return_sequences=True,
unit_forget_bias=False,
kernel_initializer =keras.initializers.Constant(bwparams['W']),
recurrent_initializer=keras.initializers.Constant(bwparams['U']),
bias_initializer=keras.initializers.Constant(bwparams['b']),
go_backwards=True,
)
backward_cell.trainable = False
birnn = layers.Bidirectional(forward_cell, backward_layer=backward_cell)
birnn.trainable = False
out = birnn(out)
return out

ws_rnn = multi_layer_birnn(ws_input, LAYERS, DIM_HIDDEN, params['birnn'])
```

to get params from Dynet I do the following
```
rnn_layers = []
for layer in dy_model.builder_layers:
fworbw = []
for fwbw in layer:
(W, U, b) = fwbw.param_collection().parameters_list()
b = b.as_array()
W = W.as_array()
U = U.as_array()
fworbw.append({
'U': U,
'W': W,
'b': b,
})
rnn_layers.append({
'fw': fworbw[0],
'bw': fworbw[1],
})
params = { 'birnn: rnn_layers }
```

I've tested and made sure that the inputs to the lstm are completely identical, however I get different output between the two frameworks :(

Contributor guide

No contributing guide indexed for this repository

Research direction

Start from the shown Dynet parameter extraction and Keras Bidirectional LSTM construction, then compare gate ordering, tensor shapes, bias layout, and backward-sequence handling. Reproduce the mismatch with identical inputs and verify that the converted weights produce matching outputs, or document the remaining framework convention difference.

Written by the indexing model from the issue text.

Assessment

Tech stack
keras, python, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.