tensorflow / tensorflow/probability

Keras VariationalGaussianProcess-Layer

Open
#919 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

keras layers
Dominant language
Jupyter Notebook
Stars
4.4k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

Hallo, I've implemented with tf a VGP-fit on my data and I would like to do the same with a keras-model and a VGP-layer. But I can't find any examples, where the kernel is a combination from several kernels and how to pass the result of tfd.VariationalGaussianProcess.optimal_variational_posterior() to the layer.

So, what I have:

variational_loc, variational_scale = (
            tfd.VariationalGaussianProcess.optimal_variational_posterior(
                mean_fn=mean_fn,
                kernel=kernel,
                inducing_index_points=inducing_index_points,
                observation_index_points=timestamp_train,
                observations=train_data.values,
                observation_noise_variance=observation_noise_variance))
observation_noise_variance = tfp.util.TransformedVariable(
                                    initial_value=1,
                                    bijector=tfb.Chain([tfb.Shift(np.float64(1e-6)), tfb.Softplus()]),
                                    dtype=dtype,
                                    name='observation_noise_variance') 
        periodic_amplitude_1y = tfp.util.TransformedVariable(
                                    initial_value=1,
                                    bijector=tfb.Chain([tfb.Shift(np.float64(1e-6)), tfb.Softplus()]),
                                    dtype=dtype,
                                    name='periodic_amplitude_1y')

        periodic_length_scale_1y = tfp.util.TransformedVariable(
                                    initial_value=1,
                                    bijector=tfb.Chain([tfb.Shift(np.float64(1e-6)), tfb.Softplus()]),
                                    dtype=dtype,
                                    name='periodic_length_scale_1y')

        periodic_period_1y = tfp.util.TransformedVariable(
                                    initial_value=24*365,
                                    bijector=tfb.Chain([tfb.Shift(np.float64(1e-6)), tfb.Softplus()]),
                                    dtype=dtype,
                                    name='periodic_period_1y')  # period of 24h * 365d

        # periodic kernel:
        periodic_kernel_1y = (tfk.ExpSinSquared(
                                    amplitude=periodic_amplitude_1y,
                                    length_scale=periodic_length_scale_1y,
                                    period=periodic_period_1y)
        )
...
        kernel = (periodic_kernel_1d * periodic_kernel_1w * periodic_kernel_1y)

And I would like to pass these to the following:

model = tf.keras.Sequential([
            tf.keras.layers.InputLayer(input_shape=[1], dtype=train_data.dtype),
            tf.keras.layers.Dense(1, kernel_initializer='ones', use_bias=False),
            tfp.layers.VariationalGaussianProcess(
                num_inducing_points=num_inducing_points,
                kernel_provider=kernel,
                event_shape=[1],
                inducing_index_points_initializer=inducing_index_points,
                variational_inducing_observations_scale_initializer=variational_scale,
                unconstrained_observation_noise_variance_initializer=observation_noise_variance,
            ),
        ])

My question ist here also, how would I pass variational_loc?

Many thanks for your help!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the tfp.layers.VariationalGaussianProcess entry point and the tfd.VariationalGaussianProcess.optimal_variational_posterior call shown in the issue. Check how kernel_provider and the variational scale initializer are expected to receive values, then determine how variational_loc is represented. Done means a documented working example covering the composite kernel and both variational parameters.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
documentation, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.