tensorflow / tensorflow/probability
Add bijectors: FillDiagonal and FillScaleDiagonal
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 4.4k
- Forks
- 1.1k
- PR merge metrics
- No merged PRs in 30d
Description
While I'd guess the use for it it would be minimal, I found myself implementing a FillDiagonal and a FillScaleDiagonal bijectors, in particular to deal with the variational_inducing_observations_scale parameter of tfd.VariationalGaussianProcess, that reads as
variational_inducing_observations_scale: `float` `Tensor`; the scale
matrix of the (full-rank Gaussian) variational posterior over function
values at the inducing points, conditional on observed data. Shape has
the form `[b1, ..., bB, e2, e2]`, where `b1, ..., bB` is broadcast
compatible with other parameters and `e2` is the number of inducing
points.
I like the general framework of VariationalGaussianProcess, even though it would be good to be able to specify a variational_inducing_observations_scale_diag parameter to avoid materialising a scale matrix while only interested in the diagonal. Side note: I briefly tried to look into the code of VariationalGaussianProcess but I was not sure how to deal with variational_gaussian_process._solve_cholesky_factored_system(cholesky_factor, rhs, name=None) when rhs is a vector representing a diagonal matrix.
While I guess this is probably a workaround to deal with the VariationalGaussianProcess case, it is probably useful for other use cases, hence if it makes sense I can submit a PR with the FillDiagonal and FillScaleDiagonal bijectors.
The bijectors are modifications of FillTriangular and FillScaleTril.
class FillDiagonal(bijector.Bijector):
"""Transforms vectors to diagonal matrices.
Given input with shape `batch_shape + [d]`, produces output with
shape `batch_shape + [d, d]`.
[...]
b = tfb.FillDiagonal()
b.forward([1, 2, 3])
# ==> [[1, 0, 0],
# [0, 2, 0],
# [0, 0, 3]]
and
class FillScaleDiagonal(chain.Chain):
"""Transforms unconstrained vectors to Diagonal matrices with positive diagonal.
This is implemented as a simple `tfb.Chain` of `tfb.FillDiagonal`
followed by `tfb.TransformDiagonal`, and provided mostly as a
convenience. The default setup is somewhat opinionated, using a
Softplus transformation followed by a small shift (`1e-5`) which
attempts to avoid numerical issues from zeros on the diagonal.
[...]
b = tfb.FillScaleDiagonal(
diag_bijector=tfb.Exp(),
diag_shift=None)
b.forward(x=[0., 0.])
# Result: [[1., 0.],
# [0., 1.]]
b.inverse(y=[[1., 0],
[0, 2]])
# Result: [log(1), log(2)]
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the existing FillTriangular and FillScaleTril bijectors, then inspect the referenced VariationalGaussianProcess usage and its _solve_cholesky_factored_system entry point. Define the forward and inverse behavior for diagonal filling and positive scale diagonals, including the proposed shift and custom diagonal bijector, and verify the examples and shape behavior with the repository's bijector tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100