google / google/edward2

Gaussian Process confronts "Cholesky decomposition was not successful. The input might not be valid."

Open
#400 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
712
Forks
77
Avg merge
9h 39m
Merged PRs (30d)
1

Description

Although [Gaussian kernel (RBF Kernel) is positive definite](https://en.wikipedia.org/wiki/Positive-definite_kernel), I think [Gaussian Process](https://github.com/google/edward2/blob/master/edward2/tensorflow/layers/gaussian_process.py) from time to time confronts
`Cholesky decomposition was not successful. The input might not be valid.` after
(1) [`covariance_matrix = self.covariance_fn(inputs, inputs)`](https://github.com/google/edward2/blob/master/edward2/tensorflow/layers/gaussian_process.py#L182) with `self.covariance_fn=ExponentiatedQuadratic`;
(2) [`covariance_matrix = tf.linalg.set_diag(covariance_matrix,tf.linalg.diag_part(covariance_matrix) + tf.keras.backend.epsilon())`](https://github.com/google/edward2/blob/master/edward2/tensorflow/layers/gaussian_process.py#L217)

Meanwhile, [`covariance_matrix = tf.linalg.set_diag(covariance_matrix,tf.linalg.diag_part(covariance_matrix) + 100* tf.keras.backend.epsilon())`](https://github.com/google/edward2/blob/master/edward2/tensorflow/layers/gaussian_process.py#L217) might help but cannot solve the problem in a scientific way,only reduce the frequency of the error:

`Cholesky decomposition was not successful. The input might not be valid.`

Contributor guide

Open the contributing guide

Research direction

Start in edward2/tensorflow/layers/gaussian_process.py around the covariance construction at lines 182 and 217, and inspect how ExponentiatedQuadratic produces the matrix before Cholesky decomposition. Reproduce the reported failure if possible and determine a principled fix rather than only increasing the diagonal epsilon; done means the failure is addressed and validated without relying on an arbitrary multiplier.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.