tensorflow / tensorflow/probability

Gaussian Process Regression in TensorFlow Probability Notebook not working

Open
#1,360 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
4.4k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

When running the tutorial notebook Gaussian Process Regression in TensorFlow Probability in Google Colab I run into
the following issue:

I execute the code cells in order, without changing any code. The cell containing the following code produces an error.

# Now we optimize the model parameters.
num_iters = 1000
optimizer = tf.optimizers.Adam(learning_rate=.01)

# Store the likelihood values during training, so we can plot the progress
lls_ = np.zeros(num_iters, np.float64)
for i in range(num_iters):
  with tf.GradientTape() as tape:
    loss = -target_log_prob(amplitude_var, length_scale_var,
                            observation_noise_variance_var)
  grads = tape.gradient(loss, trainable_variables)
  optimizer.apply_gradients(zip(grads, trainable_variables))
  lls_[i] = loss

print('Trained parameters:')
print('amplitude: {}'.format(amplitude_var._value().numpy()))
print('length_scale: {}'.format(length_scale_var._value().numpy()))
print('observation_noise_variance: {}'.format(observation_noise_variance_var._value().numpy()))

The error message is:

ValueError                                Traceback (most recent call last)
<ipython-input-81-0b42fdbef836> in <module>()
     10                             observation_noise_variance_var)
     11   grads = tape.gradient(loss, trainable_variables)
---> 12   optimizer.apply_gradients(zip(grads, trainable_variables))
     13   lls_[i] = loss
     14 

1 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/optimizer_v2/utils.py in filter_empty_gradients(grads_and_vars)
     74   if not filtered:
     75     raise ValueError("No gradients provided for any variable: %s." %
---> 76                      ([v.name for _, v in grads_and_vars],))
     77   if vars_with_empty_grads:
     78     logging.warning(

ValueError: No gradients provided for any variable: ['amplitude:0', 'length_scale:0', 'observation_noise_variance_var:0'].

After consulting Stack Overflow I tried adding tape.watch(trainable_variables), but this did not change anything.

Might someone be able to help figure out why this is happening?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the Gaussian Process Regression in TensorFlow Probability notebook in Google Colab, running the cells in order. Start at the optimization cell shown in the report and investigate why gradients for the three listed variables are empty. Done means the notebook trains and prints the parameters without the reported ValueError.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, python, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.