tensorflow / tensorflow/probability
passing a loss defined as -distr.log_prob(y) as one of the metrics
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 4.4k
- Forks
- 1.1k
- PR merge metrics
- No merged PRs in 30d
Description
Hi guys, i was trying to implement the Beta-NLL presented in https://arxiv.org/pdf/2203.09168v2.pdf .
basically the classical negative log-likelihood is multiplied by tf.stop_gradient(estimated_sigma**beta)

i tried with
def NLL2(y, distr):
return - tf.stop_gradient(distr.scale )*distr.log_prob(y)
and it seems it works but the problem is that as the paper says:
"Note that the new loss Lβ−NLL is not meant for performance evaluation, rather it is designed to result
in meaningful gradients. Due to the weighting term, the loss value does not reflect the model’s quality.
The model performance during training should be monitored with the original negative log-likelihood
objective and, optionally, with RMSE for testing the quality of the mean fit"
but when i try to pass the classical NLL
def NLL(y, distr):
return -distr.log_prob(y)
in the metrics
callback = tf.keras.callbacks.EarlyStopping(monitor='loss', patience=300)
optimizer = tf.optimizers.SGD(learning_rate=0.0002,momentum=0.9)
concrete_dropout_BNN = create_concrete_dropout_bnn_model(train_size=train_size)
concrete_dropout_BNN.compile(optimizer=optimizer,
i loss=NLL2 , jit_compile=True ,metrics= [tf.keras.metrics.RootMeanSquaredError(), NLL]
)
i get the error
AttributeError: 'Tensor' object has no attribute 'log_prob'
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the Keras compile call shown in the report and compare the inputs supplied to a metric with the distribution object expected by NLL. Reproduce the Tensor AttributeError while separating the beta-weighted loss from the original NLL metric; done when the original NLL can be monitored without that error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100