tensorflow / tensorflow/probability

AttributeError: 'SymbolicTensor' object has no attribute 'log_prob' when exporting train signature with `IndependentNormal` layer

Open
#1,790 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
4.4k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

I want to export the training signature to train my model with the C++ API. However, I am not able to export the model, even after reading through
https://github.com/tensorflow/probability/issues/742 ,
https://github.com/tensorflow/tensorflow/issues/36181
https://stackoverflow.com/questions/59743872/when-training-a-variational-bayesian-neural-network-in-tfp-how-can-i-visualize
My conda list | grep tensorflow versions are:
tensorflow 2.15.0.post1
tensorflow-estimator 2.15.0
tensorflow-io-gcs-filesystem 0.36.0
tensorflow-probability 0.23.0

My conda list | grep keras versions are:
keras 2.15.0
keras-preprocessing 1.1.2

import numpy as np
from tensorflow.keras.layers import Input, Dense
import tensorflow_probability as tfp
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import Adam
import tensorflow as tf

inputDim = 10
targetDim = 1


#build train data
samples = 1000

input_list = []
for ii in range(samples):
    input_list.append(np.arange(inputDim))

input_arr = np.array(input_list)
target = arr = np.random.normal(5.0, 0.5, (samples,1))

#define model
input = Input(shape=(inputDim))
distribution_params = Dense(2)(input)
outputs = tfp.layers.IndependentNormal(targetDim)(distribution_params)

#define loss
def nll(targets, estimated_distribution):
    return -estimated_distribution.log_prob(targets)

#compile and fit model
optimizer = Adam()
model = Model(inputs= [input] , outputs=[outputs])
model.compile(optimizer=optimizer, loss=nll)#, metrics = lossFunction)
model.summary()
model.fit(input_arr,target, shuffle=True, epochs=500)#, verbose = 2)

# test prediction
prediction = model(np.expand_dims(np.arange(inputDim), axis = 0))
print("prediction mean : ", prediction.mean())
print("stdDev = ", prediction.stddev())

#export training signature
@tf.function
def trainOp(inputs, targets):
    ### has to return loss ###
    with tf.GradientTape() as tape:
        predictions  = model(inputs)
        loss = nll(predictions, targets)
    gradients = tape.gradient(loss, model.trainable_variables)    
    optimizer.apply_gradients(zip(gradients, model.trainable_variables))
    return loss

signatures = {}
signatures["trainOp"] = trainOp.get_concrete_function(inputs = tf.TensorSpec((None, inputDim), tf.float32), 
                                                      targets = tf.TensorSpec((None, targetDim), tf.float32))

model.save('./testExport/', save_traces = False, signatures = signatures)

failes with :
AttributeError: in user code:

File "/home/aberberich/Shared/Andi/tf2Api/reworked/min_export_failure.py", line 57, in trainOp  *
    loss = nll(predictions, targets)
File "/home/aberberich/Shared/Andi/tf2Api/reworked/min_export_failure.py", line 37, in nll  *
    return -estimated_distribution.log_prob(targets)

AttributeError: 'SymbolicTensor' object has no attribute 'log_prob'`

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reproduction in min_export_failure.py, especially trainOp and nll around the reported lines, then inspect how model.save exports the trainOp concrete function. Reproduce with the listed TensorFlow and TensorFlow Probability versions and determine what change allows the training signature to export without the SymbolicTensor log_prob error.

Written by the indexing model from the issue text.

Assessment

Tech stack
machine-learning, python
Domain
api, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.