tensorflow / tensorflow/probability

Inverse Autoregressive Flow (IAF) "symbolic Keras input/output" TypeError?

Open
#1,437 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
4.4k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

In the AutoregressiveNetwork documentation

https://github.com/tensorflow/probability/blob/fb99b12772513dc6888ea0854a4a891b4bbb4b21/tensorflow_probability/python/bijectors/masked_autoregressive.py#L652-L669

following

# Generate data -- as in Figure 1 in [Papamakarios et al. (2017)][2]).
n = 2000
x2 = np.random.randn(n).astype(dtype=np.float32) * 2.
x1 = np.random.randn(n).astype(dtype=np.float32) + (x2 * x2 / 4.)
data = np.stack([x1, x2], axis=-1)

# Density estimation with MADE.
made = tfb.AutoregressiveNetwork(params=2, hidden_units=[10, 10])

this code works

distribution = tfd.TransformedDistribution(
    distribution=tfd.Sample(tfd.Normal(loc=0., scale=1.), sample_shape=[2]),
    bijector=tfb.MaskedAutoregressiveFlow(made))

# Construct and fit model.
x_ = tfkl.Input(shape=(2,), dtype=tf.float32)
log_prob_ = distribution.log_prob(x_)
model = tfk.Model(x_, log_prob_)

but if tfb.Invert is added as is done in the MaskedAutoregressiveFlow documentation

https://github.com/tensorflow/probability/blob/fb99b12772513dc6888ea0854a4a891b4bbb4b21/tensorflow_probability/python/bijectors/masked_autoregressive.py#L164-L188

then the code fails

distribution = tfd.TransformedDistribution(
    distribution=tfd.Sample(tfd.Normal(loc=0., scale=1.), sample_shape=[2]),
    bijector=tfb.Invert(tfb.MaskedAutoregressiveFlow(made))) # Here is the only change from previously working code

# Construct and fit model.
x_ = tfkl.Input(shape=(2,), dtype=tf.float32)
log_prob_ = distribution.log_prob(x_)
model = tfk.Model(x_, log_prob_)

with this error message

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-11-3e5fc03eaed6> in <module>()
      9 # Construct and fit model.
     10 z1_ = tfkl.Input(shape=(2,), dtype=tf.float32)
---> 11 log_prob_ = f_z1.log_prob(z1_)
     12 model = tfk.Model(z1_, log_prob_)
     13 

20 frames

/usr/local/lib/python3.7/dist-packages/keras/engine/keras_tensor.py in __array__(self)
    243   def __array__(self):
    244     raise TypeError(
--> 245         'Cannot convert a symbolic Keras input/output to a numpy array. '
    246         'This error may indicate that you\'re trying to pass a symbolic value '
    247         'to a NumPy call, which is not supported. Or, '

TypeError: Cannot convert a symbolic Keras input/output to a numpy array. This error may indicate that you're trying to pass a symbolic value to a NumPy call, which is not supported. Or, you may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model.

Does anyone have any experience with this issue or workarounds for implementing an inverse autoregressive flow (IAF) within the keras framework?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the AutoregressiveNetwork and MaskedAutoregressiveFlow documentation examples linked in the issue, then reproduce the two Keras models using a symbolic tfkl.Input. Compare the working flow with the version wrapped in tfb.Invert and trace the TypeError; done means the inverse autoregressive flow can be used in the Keras model without that symbolic-input failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
keras, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.