google / google/edward2

`gp_kernel_scale_trainable=True` in RandomFeatureGaussianProcess does not work

Open
#419 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
712
Forks
77
Avg merge
9h 39m
Merged PRs (30d)
1

Description

Setting [`gp_kernel_scale_trainable=True`](https://github.com/google/edward2/blob/311d3ad6946b543e70af1495eab9a0a9b4f69854/edward2/tensorflow/layers/random_feature.py#L54) in [`class RandomFeatureGaussianProcess(`](https://github.com/google/edward2/blob/311d3ad6946b543e70af1495eab9a0a9b4f69854/edward2/tensorflow/layers/random_feature.py#L47) does not work. Anyway, we might have hidden layers for raw inputs/features.

I also try to modify:
```diff
self.unscaled_kernel = self.add_weight(
name='unscaled_kernel',
shape=(input_dim, self.output_dim),
dtype=dtypes.float32,
initializer=kernel_initializer,
++ trainable=True)
```
and pay attention to
```if self.scale is None:
self.scale = _get_default_scale(self.kernel_initializer, input_dim)
self.kernel_scale = self.add_weight(
name='kernel_scale',
shape=(1,),
dtype=dtypes.float32,
initializer=init_ops.constant_initializer(self.scale),
trainable=True,
constraint='NonNeg')
```
in [`tf.keras.layers.experimental.RandomFourierFeatures` (the file is `kernelized.py`)](https://github.com/tensorflow/tensorflow/blob/v2.3.1/tensorflow/python/keras/layers/kernelized.py#L40-L247)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.