google-deepmind / google-deepmind/sonnet
v2: `dynamic_unroll` using `lstm_with_recurrent_dropout` doesn't work
- Dominant language
- Python
- Stars
- 10k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
The following code doesn't work:
```
import tensorflow as tf
import numpy as np
import sonnet as snt
class A():
def __init__(self):
self.train_core, test_core = snt.lstm_with_recurrent_dropout(1, dropout=0.5)
@tf.function
def forward(self, inp):
return snt.dynamic_unroll(self.train_core, inp, self.train_core.initial_state(5))
a = A()
inp = tf.tile(tf.linspace(-1., 1., 20)[:,None,None], [1,5,1])
print(a.forward(inp))
```
promopting
```
ValueError: in converted code:
test.py:13 forward *
return snt.dynamic_unroll(self.train_core, inp,
/home/ziyu/sonnet/sonnet/src/utils.py:310 smart_autograph_wrapper *
return f_autograph(*args, **kwargs)
/tmp/tmp8pmfpnpt.py:79 tf__dynamic_unroll
state, output_tas, outputs = ag__.for_stmt(ag__.converted_call(tf.range, (1, num_steps), None, fscope), None, loop_body, get_state, set_state, (state, output_tas, outputs), ('state', 'output_tas', 'outputs'), ())
/home/ziyu/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow_core/python/autograph/operators/control_flow.py:315 for_stmt
composite_symbol_names)
/home/ziyu/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow_core/python/autograph/operators/control_flow.py:478 _tf_range_for_stmt
opts=opts,
/home/ziyu/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow_core/python/autograph/operators/control_flow.py:794 _tf_while_stmt
aug_init_vars, **opts)
/home/ziyu/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow_core/python/ops/control_flow_ops.py:2675 while_loop
back_prop=back_prop)
/home/ziyu/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow_core/python/ops/while_v2.py:77 while_loop
expand_composites=True)
/home/ziyu/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow_core/python/util/nest.py:568 map_structure
structure[0], [func(*x) for x in entries],
/home/ziyu/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow_core/python/util/nest.py:568
structure[0], [func(*x) for x in entries],
/home/ziyu/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow_core/python/framework/indexed_slices.py:318 internal_convert_to_tensor_or_indexed_slices
return ops.convert_to_tensor(value, dtype=dtype, name=name, as_ref=as_ref)
/home/ziyu/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py:1314 convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
/home/ziyu/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow_core/python/framework/constant_op.py:317 _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
/home/ziyu/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow_core/python/framework/constant_op.py:258 constant
allow_broadcast=True)
/home/ziyu/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow_core/python/framework/constant_op.py:296 _constant_impl
allow_broadcast=allow_broadcast))
/home/ziyu/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow_core/python/framework/tensor_util.py:439 make_tensor_proto
raise ValueError("None values not supported.")
ValueError: None values not supported.
```
Seems it is because you are trying to store a `None` value to the initial state of a while loop. Changing the None value on [this line](https://github.com/deepmind/sonnet/blob/1f5c0c241653ef692f8a5c4385d6ea5e5e44bdef/sonnet/src/recurrent.py#L1201) to `0.` appears to fix this.
I'm using `tensorflow-gpu 2.1.0` and python 3.6. For sonnet I've tried both `2.0.0b0` and the HEAD version (`1f5c0c241`).
Contributor guide
Assessment
This issue has not been assessed yet.