tensorflow / tensorflow/probability

Document that tfp.distributions.MixtureSameFamily uses softmax on the fractions

Open
#1,727 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
4.4k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

Hello, it tooks me a whlie to understand this:

my_pdf = tfp.layers.MixtureNormal(3, 1)(np.array([1/3, 0/3, 2/3,
                                 4,  # mean1
                                 5,  # std1
                                 6,  # mean2
                                 7,  # std2
                                 8,  # mean3
                                 9   # std3
                                 ], dtype=float))
print(my_pdf.mixture_distribution.probs_parameter())
print(my_pdf.components_distribution.mean().numpy())
print(my_pdf.components_distribution.variance().numpy())
tf.Tensor([0.3213219  0.23023722 0.44844085], shape=(3,), dtype=float32)
[[4.]
 [6.]
 [8.]]
[[25.067198]
 [49.012764]
 [81.00221 ]]

Where the first three numbers are coming from? Why they are not 1/3, 0/3, 2/3? Then I look to the code and found this

https://github.com/tensorflow/probability/blob/main/tensorflow_probability/python/distributions/mixture_same_family.py#LL343C20-L343C39

Why a softmax is applied to the fractions? E.g.

np.exp(tf.math.log_softmax([1/3, 0/3, 2/3]))

I am not sure why this is done, but please can you document it?
I undestand you want the numbers to sum to 1, but I guess this can be requested.

In addition I have recently discovered that also a softsum is applied to the stds

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked code in tensorflow_probability/python/distributions/mixture_same_family.py around line 343 and reproduce the issue's MixtureNormal example. Document how the mixture fractions and component scale values are transformed, with wording that explains the observed outputs and makes the expected parameterization clear.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.