Gaussian Process confronts "Cholesky decomposition was not successful. The input might not be valid."
- Dominant language
- Jupyter Notebook
- Stars
- 712
- Forks
- 77
- Avg merge
- 9h 39m
- Merged PRs (30d)
- 1
Description
Although [Gaussian kernel (RBF Kernel) is positive definite](https://en.wikipedia.org/wiki/Positive-definite_kernel), I think [Gaussian Process](https://github.com/google/edward2/blob/master/edward2/tensorflow/layers/gaussian_process.py) from time to time confronts
`Cholesky decomposition was not successful. The input might not be valid.` after
(1) [`covariance_matrix = self.covariance_fn(inputs, inputs)`](https://github.com/google/edward2/blob/master/edward2/tensorflow/layers/gaussian_process.py#L182) with `self.covariance_fn=ExponentiatedQuadratic`;
(2) [`covariance_matrix = tf.linalg.set_diag(covariance_matrix,tf.linalg.diag_part(covariance_matrix) + tf.keras.backend.epsilon())`](https://github.com/google/edward2/blob/master/edward2/tensorflow/layers/gaussian_process.py#L217)
Meanwhile, [`covariance_matrix = tf.linalg.set_diag(covariance_matrix,tf.linalg.diag_part(covariance_matrix) + 100* tf.keras.backend.epsilon())`](https://github.com/google/edward2/blob/master/edward2/tensorflow/layers/gaussian_process.py#L217) might help but cannot solve the problem in a scientific way,only reduce the frequency of the error:
`Cholesky decomposition was not successful. The input might not be valid.`
Contributor guide
Research direction
Start in edward2/tensorflow/layers/gaussian_process.py around the covariance construction at lines 182 and 217, and inspect how ExponentiatedQuadratic produces the matrix before Cholesky decomposition. Reproduce the reported failure if possible and determine a principled fix rather than only increasing the diagonal epsilon; done means the failure is addressed and validated without relying on an arbitrary multiplier.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100