tensorflow / tensorflow/probability

Unexpected Symbolic tensor in Tensorflow Probability tensor_coercible object (mixture layer)

Open
#1,806 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
4.4k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

Complex interpretation of Tensorflow Probability tensor_coercible object
I have a tensorflow model (keras sequential) that ends with a Tensorflow Probability (TFP) mixture layer. My goal is to fit this network with a custom loss function. The unexpected behaviour is the following:

When I pass a custom loss like this:

def wass_loss(y_true, y_pred):
    print(type(y_pred))
    print(type(y_true))
    # Do multiple 'tf' operations, one of them, sample the _TensorCoercible
    return ...

It prints:

<class 'tensorflow_probability.python.layers.internal.distribution_tensor_coercible._TensorCoercible'>
<class 'tensorflow.python.framework.ops.SymbolicTensor'>

Intepreting each argument correctly and results are ok /coherent.

Now, when I compile this same network with another loss func. and use the wass_loss above as metric.

def another_loss(y_true, y_pred):
    return -y_pred.log_prob(y_true)

my_model.compile(optimizer = kr.optimizers.Adam(1e-3),
                 loss = another_loss,
                 metrics = [wass_loss,])
my_model.fit(...)

I get:

<class 'tensorflow.python.framework.ops.SymbolicTensor'>
<class 'tensorflow.python.framework.ops.SymbolicTensor'>

Which is unexpected, I would expect the Tensor Coercible from TFP to still be a Tensor Coercible. Yet, looks like that after the new loss (which might be computed first), it is already a Symbolic Tensor Why is it happening? What am I missing?

Thanks a lot!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the behavior with the Keras Sequential model, TensorFlow Probability mixture layer, and the two compile/fit configurations shown in the issue. Compare the custom loss and metric inputs, then document the cause and expected behavior; no source file or test is named in the report.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.