tensorflow / tensorflow/probability

Feature request: Random variable (distribution object) as result of mathematical functions of random variables (distributions)

Open
#1,157 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
4.4k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

Simple example that should have been possible:
X ~ Uniform(a,b)
Y ~ Uniform(c,d)
X,Y are independent. Then:
Z = X + Y
has some trapezoidoid (🙃) distribution over (a+c,b+d).

I would like to be able to sample Z directly. I would also like to read probability densities, interval probabilities, etc. from the Z distribution object. If a,b,c,d are trainable variables, I would like to be able to optimize them for a Maximum Likelihood Objective obtained by querying the Z distribution object with batches of targets.

This is obviously a very simple example. Ideally, I would be able to define distributions by applying (almost) arbitrary functions to multiple distribution objects, and seamlessly use the resulting distribution objects as I would use the built-in distributions. Regardless of the number of nesting levels, everything would be trainable end-to-end, to the extent the constituent functions and distributions were trainable.

I must admit I do not know how _tfp.distribution_s are trained now (although I have magically succeeded both via samples and via log-density), so I understand there is a possibility that my wishes are unrealistic. I have done my best trying to find documentation, blogs and web questions on how to combine random variables like this, with no success. Mixtures and joint distributions combine distribution probabilities and the joint sequential distribution allows building a sort of chaining graph, but to my knowledge none of the distribution classes offer a way to apply math functions to random variables represented as distribution objects and treat the resulting random variable as a distribution object.

Here is a minimal example program based on the example, trying (naïvely?) to add the variables directly:

import tensorflow as tf
import tensorflow_probability as tfp

X = tfp.distributions.Uniform(2, 3)
Y = tfp.distributions.Uniform(-1,2)

Z = X+Y

And here is the console output:

Traceback (most recent call last):
  File ".\dist_sum_test.py", line 7, in <module>
    Z = X+Y
TypeError: unsupported operand type(s) for +: 'Uniform' and 'Uniform'

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the minimal TensorFlow Probability example in the issue and review how mixtures, joint distributions, and existing distribution APIs represent composition. Done would require a settled scope for transformed multi-distribution objects, including sampling, density or probability queries, and trainable parameters; the issue names no files or tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
backend-api-design, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.