Is it possible to have a mixture of distributions as the prior of a Bayesian layer?

The paper Weight Uncertainty in Neural Networks proposes the usage of a mixture of two Gaussians as the prior distribution. By default, the priors (of the kernel and bias) of a Convolution2DFlipout layer are Gaussians. Is there an easy way of having a mixture of Gaussians as the priors?

I know TFP provides the class MixtureSameFamily. For example, you can create a mixture of two Gaussians in the following way.

from tensorflow_probability import distributions as tfd

prior = tfd.MixtureSameFamily(tfd.Categorical(probs=[0.4, 0.6]),
                              components_distribution=tfd.Normal(
                                  loc=[-1., 1],  # One for each component.
                                  scale=[0.1, 0.5])
                              )

In order to have a mixture of two Gaussians as the prior of a Bayesian layer, there must be a way of computing the KL divergence between a mixture of two Gaussians and another Gaussian (the posterior). After having had looked at the documentation of tfp.distributions.kl_divergence, apparently, this is not yet implemented.

Start with the Convolution2DFlipout Bayesian layer and tfp.distributions.kl_divergence, then review how MixtureSameFamily represents the proposed prior. Done means a mixture-of-Gaussians prior can be used with the layer and its KL divergence against the Gaussian posterior is supported.