tensorflow / tensorflow/probability

Conditional AutoregressiveNetwork doesn't work with tfb.Chain

Open
#1,410 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
4.4k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

Dear all,
I am trying to implement a conditional MAF based on the example provided. It works fine when there is only one bijector used in TransformedDistribution, but as soon as a tfb.Chain is used it breaks due to not passing the bijector_kwargs correctly through the chain (I assume this is the issue). The error thrown is "ValueError: conditional_input must be passed as a named argument".
I am aware of the issue #1159, however I am trying to pass a distribution object through the chained bijector via a transformed distribution, so I don't think the solution given there applies. I am not sure if I try to do something that isn't meant to be done, if this is a bug, or if I simply do it wrong.

import numpy as np
import tensorflow as tf
import tensorflow_probability as tfp
tfb = tfp.bijectors
tfd = tfp.distributions
tfkl = tf.keras.layers
tfk = tf.keras

n = 2000
c = np.r_[
np.zeros(n//2),
np.ones(n//2)
]
mean_0, mean_1 = 0, 5
x = np.r_[
np.random.randn(n//2).astype(dtype=np.float32) + mean_0,
np.random.randn(n//2).astype(dtype=np.float32) + mean_1
]

made0 = tfb.AutoregressiveNetwork(params=2, hidden_units=[2, 2], event_shape=(1,),
conditional=True, kernel_initializer=tfk.initializers.VarianceScaling(0.1),
conditional_event_shape=(1,)
)
made1 = tfb.AutoregressiveNetwork(params=2, hidden_units=[2, 2], event_shape=(1,),
conditional=True, kernel_initializer=tfk.initializers.VarianceScaling(0.1),
conditional_event_shape=(1,)
)

tot_bijector = tfb.Chain([tfb.MaskedAutoregressiveFlow(made0), tfb.MaskedAutoregressiveFlow(made1)])

distribution = tfd.TransformedDistribution(
distribution=tfd.Sample(tfd.Normal(loc=0., scale=1.), sample_shape=[1]),
bijector=tot_bijector)

x_ = tfkl.Input(shape=(1,), dtype=tf.float32)
c_ = tfkl.Input(shape=(1,), dtype=tf.float32)
log_prob_ = distribution.log_prob(
x_, bijector_kwargs={'conditional_input': c_})
model = tfk.Model([x_, c_], log_prob_)

model.compile(optimizer=tf.optimizers.Adam(learning_rate=0.1),
loss=lambda _, log_prob: -log_prob)

batch_size = 25
model.fit(x=[x, c],
y=np.zeros((n, 0), dtype=np.float32),
batch_size=batch_size,
epochs=3,
steps_per_epoch=n // batch_size,
shuffle=True,
verbose=True)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the supplied reproduction with AutoregressiveNetwork, MaskedAutoregressiveFlow, tfb.Chain, and TransformedDistribution. Trace how bijector_kwargs and conditional_input are forwarded through the chain; done means the conditional chained distribution no longer raises the named-argument ValueError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.