stan-dev / stan-dev/math

Add low-rank multivariate normal distribution

Open
#3,288 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
839
Forks
220
Avg merge
2d 4h
Merged PRs (30d)
14

Description

Description

If a low-rank structure exists (or is assumed to exist) in the covariance matrix of MVN, i.e., $$\Sigma=W W^\top + D$$, where $$W$$ is $$n\times m$$ ($$m<<n$$), $$D$$ is a positive definite diagonal matrix, then the determinant and the inverse can be reduced from $$O(n^3)$$ to $$O(nm^2+m^3)$$. A continuation of #3038.

For implementation, can refer to
https://en.wikipedia.org/wiki/Woodbury_matrix_identity
https://en.wikipedia.org/wiki/Matrix_determinant_lemma

numpyro
Low-rank Multivariate Normal distribution · Issue #387 · pyro-ppl/numpyro
https://github.com/pyro-ppl/numpyro/blob/master/numpyro/distributions/continuous.py

pytorch
Probability distributions - torch.distributions — PyTorch 2.10 documentation
pytorch/torch/distributions/lowrank_multivariate_normal.py at 14e348b7ad1b3472812f2b077020d80deaf6a787 · pytorch/pytorch

Applications:
Intermediate between mean-field and full-rank ADVI
AutoLowRankMultivariateNormal-numpyro

Expected Output

Maybe follow the multi_normal_XXX series?

real multi_normal_lowrank_lpdf(vector y | vector mu, matrix cov_factor, vector cov_diag)
also
real multi_normal_lowrank_lpdf(vectors y | vectors mu, matrix cov_factor, vector cov_diag)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by examining the existing multi_normal_XXX distribution series in Stan Math and compare the referenced NumPyro continuous.py and PyTorch lowrank_multivariate_normal.py implementations. Use the Woodbury matrix identity and matrix determinant lemma references to define the low-rank covariance behavior, with completion demonstrated by the two scalar and vectorized multi_normal_lowrank_lpdf signatures described in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.