tensorflow / tensorflow/probability
Inverse Autoregressive Flow (IAF) "symbolic Keras input/output" TypeError?
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 4.4k
- Forks
- 1.1k
- PR merge metrics
- No merged PRs in 30d
Description
In the AutoregressiveNetwork documentation
following
# Generate data -- as in Figure 1 in [Papamakarios et al. (2017)][2]).
n = 2000
x2 = np.random.randn(n).astype(dtype=np.float32) * 2.
x1 = np.random.randn(n).astype(dtype=np.float32) + (x2 * x2 / 4.)
data = np.stack([x1, x2], axis=-1)
# Density estimation with MADE.
made = tfb.AutoregressiveNetwork(params=2, hidden_units=[10, 10])
this code works
distribution = tfd.TransformedDistribution(
distribution=tfd.Sample(tfd.Normal(loc=0., scale=1.), sample_shape=[2]),
bijector=tfb.MaskedAutoregressiveFlow(made))
# Construct and fit model.
x_ = tfkl.Input(shape=(2,), dtype=tf.float32)
log_prob_ = distribution.log_prob(x_)
model = tfk.Model(x_, log_prob_)
but if tfb.Invert is added as is done in the MaskedAutoregressiveFlow documentation
then the code fails
distribution = tfd.TransformedDistribution(
distribution=tfd.Sample(tfd.Normal(loc=0., scale=1.), sample_shape=[2]),
bijector=tfb.Invert(tfb.MaskedAutoregressiveFlow(made))) # Here is the only change from previously working code
# Construct and fit model.
x_ = tfkl.Input(shape=(2,), dtype=tf.float32)
log_prob_ = distribution.log_prob(x_)
model = tfk.Model(x_, log_prob_)
with this error message
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-11-3e5fc03eaed6> in <module>()
9 # Construct and fit model.
10 z1_ = tfkl.Input(shape=(2,), dtype=tf.float32)
---> 11 log_prob_ = f_z1.log_prob(z1_)
12 model = tfk.Model(z1_, log_prob_)
13
20 frames
/usr/local/lib/python3.7/dist-packages/keras/engine/keras_tensor.py in __array__(self)
243 def __array__(self):
244 raise TypeError(
--> 245 'Cannot convert a symbolic Keras input/output to a numpy array. '
246 'This error may indicate that you\'re trying to pass a symbolic value '
247 'to a NumPy call, which is not supported. Or, '
TypeError: Cannot convert a symbolic Keras input/output to a numpy array. This error may indicate that you're trying to pass a symbolic value to a NumPy call, which is not supported. Or, you may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model.
Does anyone have any experience with this issue or workarounds for implementing an inverse autoregressive flow (IAF) within the keras framework?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the AutoregressiveNetwork and MaskedAutoregressiveFlow documentation examples linked in the issue, then reproduce the two Keras models using a symbolic tfkl.Input. Compare the working flow with the version wrapped in tfb.Invert and trace the TypeError; done means the inverse autoregressive flow can be used in the Keras model without that symbolic-input failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- keras, tensorflow
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100