tensorflow / tensorflow/probability

Add bijectors: FillDiagonal and FillScaleDiagonal

Open
#791 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
4.4k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

While I'd guess the use for it it would be minimal, I found myself implementing a FillDiagonal and a FillScaleDiagonal bijectors, in particular to deal with the variational_inducing_observations_scale parameter of tfd.VariationalGaussianProcess, that reads as

variational_inducing_observations_scale: `float` `Tensor`; the scale
        matrix of the (full-rank Gaussian) variational posterior over function
        values at the inducing points, conditional on observed data. Shape has
        the form `[b1, ..., bB, e2, e2]`, where `b1, ..., bB` is broadcast
        compatible with other parameters and `e2` is the number of inducing
        points.

I like the general framework of VariationalGaussianProcess, even though it would be good to be able to specify a variational_inducing_observations_scale_diag parameter to avoid materialising a scale matrix while only interested in the diagonal. Side note: I briefly tried to look into the code of VariationalGaussianProcess but I was not sure how to deal with variational_gaussian_process._solve_cholesky_factored_system(cholesky_factor, rhs, name=None) when rhs is a vector representing a diagonal matrix.

While I guess this is probably a workaround to deal with the VariationalGaussianProcess case, it is probably useful for other use cases, hence if it makes sense I can submit a PR with the FillDiagonal and FillScaleDiagonal bijectors.
The bijectors are modifications of FillTriangular and FillScaleTril.

class FillDiagonal(bijector.Bijector):
    """Transforms vectors to diagonal matrices.

  Given input with shape `batch_shape + [d]`, produces output with
  shape `batch_shape + [d, d]`.
  [...]
  b = tfb.FillDiagonal()
  b.forward([1, 2, 3])
  # ==> [[1, 0, 0],
  #      [0, 2, 0],
  #      [0, 0, 3]]

and

class FillScaleDiagonal(chain.Chain):
    """Transforms unconstrained vectors to Diagonal matrices with positive diagonal.
  This is implemented as a simple `tfb.Chain` of `tfb.FillDiagonal`
  followed by `tfb.TransformDiagonal`, and provided mostly as a
  convenience. The default setup is somewhat opinionated, using a
  Softplus transformation followed by a small shift (`1e-5`) which
  attempts to avoid numerical issues from zeros on the diagonal.
  [...]
  b = tfb.FillScaleDiagonal(
       diag_bijector=tfb.Exp(),
       diag_shift=None)
  b.forward(x=[0., 0.])
  # Result: [[1., 0.],
  #          [0., 1.]]
  b.inverse(y=[[1., 0],
               [0, 2]])
  # Result: [log(1), log(2)]

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the existing FillTriangular and FillScaleTril bijectors, then inspect the referenced VariationalGaussianProcess usage and its _solve_cholesky_factored_system entry point. Define the forward and inverse behavior for diagonal filling and positive scale diagonals, including the proposed shift and custom diagonal bijector, and verify the examples and shape behavior with the repository's bijector tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.