SUG 9.5 suggests a degenerate model
Nobody has claimed this yet.
- Dominant language
- TeX
- Stars
- 43
- Forks
- 133
- Avg merge
- 11h 32m
- Merged PRs (30d)
- 4
Description
In section 9.5, the full Bayes for the correlated topic model uses a mvnormal variate transformed by a softmax as probabilities. The problem is that the inverse softmax doesn't work as a link function because it isn't unique. My impression from a brief perusal of Aitchison & Shen 1980 is that the solution is to fix one element of the link-scale parameter vector to zero, which identifies which unique inverse-softmax we're dealing with as our link function.
The model of Blei & Lafferty (also cited in the SUG) appears to circumvent the problem by using the maximum a posteriori estimate for the link-scale mvnormal. Although the posterior is degenerate, it still has a maximum thanks to the prior, and once the estimate is fixed to that maximum the degeneracy disappears.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading section 9.5 and the cited Aitchison & Shen and Blei & Lafferty references. Verify whether the inverse-softmax parameterization is identifiable and determine the correct documentation change; done means the section no longer presents a degenerate model without an explicit resolution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- tex
- Domain
- documentation
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100