tensorflow / tensorflow/probability
Shared Covariance in MultivariateNormalTriL?
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 4.4k
- Forks
- 1.1k
- PR merge metrics
- No merged PRs in 30d
Description
I'm building a Linear Discriminant Analysis model using tfp. My feature size is 1280, with 100 output dimensions (classes).
I am able to calculate per-class $\mu$ and a covariance matrix $\Sigma$ shared across all classes.
This is what my distribution object looks like:
# class_means = [100, 1280]
# shared_cov = [1280, 1280]
dist = tfp.distributions.MultivariateNormalTriL(
loc=class_means,
scale_tril=tf.linalg.cholesky(shared_cov)
)
Problem: I have a test dataset of features, batched such that each element of batch has a shape [bs, 1280].
Looping over batches dist.log_prob(tf.expand_dims(x, 1)) yields correct accuracy, but is prohibitively slow.
I'm concerned if the shared_cov is broadcasted, inverted and computed for log_prob of each and every class. I have a custom implementation in tensorflow that does all the low level operations which is significantly faster (since it inverts covariance only once, and is reused across all classes)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked Colab reproduction and the MultivariateNormalTriL.log_prob path used by dist.log_prob(tf.expand_dims(x, 1)). Check whether the shared scale_tril is recomputed or inverted for each class, then compare any change against the custom TensorFlow implementation and the reported accuracy and runtime. Done means the shared covariance computation is reused without changing results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook, python
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100