tensorflow / tensorflow/probability
Redesign of layers.util.default_multivariate_normal_fn
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 4.4k
- Forks
- 1.1k
- PR merge metrics
- No merged PRs in 30d
Description
Whilst playing around with the variational layers, I was trying to specify a prior using the default_multivariate_normal_fn. With the current design it isn't really possible (as far as I could tell).
It looks like you could use the default_mean_field_normal_fn, though if using as a prior within a variational layer there isn't a nice way to specify that the loc and scale parameters aren't trainable, as the parameters are initialised with the layers trainable attribute eg. like here.
I have fixed this myself by re-implementing the function to be similar to the default_mean_field_fn function, and that it simply forces the trainable argument in sub function _fn here to be False. Whilst doing this I set all the default arguments to mimic the original default_multivariate_normal_fn.
I will open a PR for this soon, but just wanted to get some feedback on whether this is something other people/the TFP team are interested in or if they have a better solution.
Another solution could be to add a trainable_prior attribute to the variational layers, instead of the priors being initialised with the layers general trainable attribute. This is a bit more general and I think could raise issues for others who aren't sure why they are computing gradients w.r.t. their priors, so might be worth opening another issue but I would also like to hear what people think about this.
Thanks :)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read tensorflow_probability/python/layers/util.py around default_multivariate_normal_fn and default_mean_field_normal_fn, then inspect tensorflow_probability/python/layers/dense_variational.py where prior parameters use the layer's trainable attribute. First clarify whether the project wants a redesigned prior function or a trainable_prior attribute. Done means the chosen API supports non-trainable prior parameters without ambiguity.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100