tensorflow / tensorflow/probability
keras JointDistribution minimal example not working
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 4.4k
- Forks
- 1.1k
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
I am trying to train a keras model using a mix of JointDistribution and other layers. My goal is to be able to build the model, compile and use model.fit on data.
Somehow, I keep getting errors and can't get it working.
I could not find a minimal example with it running and I think I am missing something.
Here is the minimal example I tried:
import tensorflow as tf
import tensorflow_probability as tfp
import tf_keras as tfk
import numpy as np
tfd = tfp.distributions
tfpl = tfp.layers
# Generate synthetic data
np.random.seed(42)
x = np.random.randn(2000, 1).astype(np.float32)
true_w, true_b = 2.0, -1.0
y = true_w * x + true_b + np.random.randn(2000, 1).astype(np.float32) * 0.5
# Define the JointDistributionNamed model
def make_joint(x):
return tfd.JointDistributionNamed({
"weight": tfd.Normal(loc=0., scale=1.),
"bias": tfd.Normal(loc=0., scale=1.),
"obs": lambda weight, bias: tfd.Independent(
tfd.Normal(loc=weight * x + bias, scale=0.5),
reinterpreted_batch_ndims=1
)
})
# Keras model using DistributionLambda
inputs = tfk.Input(shape=(1,))
dense = tfk.layers.Dense(2)(inputs) # Not used for parameterization here, just as a placeholder
def posterior_fn(params):
# For demonstration, use fixed x from outer scope
jd = make_joint(x)
return jd
outputs = tfpl.DistributionLambda(posterior_fn)(dense)
model = tfk.Model(inputs=inputs, outputs=outputs)
# Custom loss: negative log-likelihood of observed y under the joint
def nll(y_true, y_pred):
# y_pred is a JointDistributionNamed, so evaluate log_prob of obs
return -y_pred.log_prob({"obs": y_true})
model.compile(optimizer=tf.optimizers.Adam(0.01), loss=nll)
model.fit(x, y, epochs=5, batch_size=32)
And I got this error:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
[<ipython-input-1-e7f1317b9e9b>](https://localhost:8080/#) in <cell line: 0>()
33 return jd
34
---> 35 outputs = tfpl.DistributionLambda(posterior_fn)(dense)
36 model = tfk.Model(inputs=inputs, outputs=outputs)
37
3 frames
[/usr/local/lib/python3.11/dist-packages/tensorflow_probability/python/layers/distribution_layer.py](https://localhost:8080/#) in _fn(*fargs, **fkwargs)
183 # TODO(b/126056144): Remove silent handle once we identify how/why Keras
184 # is losing the distribution handle for activity_regularizer.
--> 185 value._tfp_distribution = distribution # pylint: disable=protected-access
186 # TODO(b/120153609): Keras is incorrectly presuming everything is a
187 # `tf.Tensor`. Closing this bug entails ensuring Keras only accesses
AttributeError: Exception encountered when calling layer "distribution_lambda" (type DistributionLambda).
'dict' object has no attribute '_tfp_distribution'
Call arguments received by layer "distribution_lambda" (type DistributionLambda):
• inputs=tf.Tensor(shape=(None, 2), dtype=float32)
• args=<class 'inspect._empty'>
• kwargs={'training': 'None'}
I am posting it here because I feel this might be a bug.
You can reproduce it via this colab link
Please note I also tried with tfd.JointDistributionSequential and no luck either.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the linked Colab using the JointDistributionNamed example and inspect tensorflow_probability/python/layers/distribution_layer.py at the DistributionLambda call shown in the traceback. Compare the expected distribution return with the observed dict error, and confirm whether a working model.fit path or a documented limitation is needed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100