tensorflow / tensorflow/probability

Shared Covariance in MultivariateNormalTriL?

Open
#1,662 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
4.4k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

I'm building a Linear Discriminant Analysis model using tfp. My feature size is 1280, with 100 output dimensions (classes).
I am able to calculate per-class $\mu$ and a covariance matrix $\Sigma$ shared across all classes.

This is what my distribution object looks like:

# class_means = [100, 1280]
# shared_cov  = [1280, 1280]
dist = tfp.distributions.MultivariateNormalTriL(
    loc=class_means, 
    scale_tril=tf.linalg.cholesky(shared_cov)
)

Problem: I have a test dataset of features, batched such that each element of batch has a shape [bs, 1280].
Looping over batches dist.log_prob(tf.expand_dims(x, 1)) yields correct accuracy, but is prohibitively slow.

I'm concerned if the shared_cov is broadcasted, inverted and computed for log_prob of each and every class. I have a custom implementation in tensorflow that does all the low level operations which is significantly faster (since it inverts covariance only once, and is reused across all classes)

Edit: Reproduce the issue Open In Colab

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked Colab reproduction and the MultivariateNormalTriL.log_prob path used by dist.log_prob(tf.expand_dims(x, 1)). Check whether the shared scale_tril is recomputed or inverted for each class, then compare any change against the custom TensorFlow implementation and the reported accuracy and runtime. Done means the shared covariance computation is reused without changing results.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, python
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.