scikit-learn / scikit-learn/scikit-learn

LatentDirichletAllocation's components aren't normalized

Open
#6,353 20 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Enhancement module:decomposition Needs Decision
Dominant language
Python
Stars
67.3k
Forks
27.4k
Avg merge
1d 15h
Merged PRs (30d)
58

Description

An LDA model's components_ isn't easily interpretable:

>>> X = np.abs(np.random.RandomState(42).randn(5, 4))
>>> from sklearn.decomposition import LatentDirichletAllocation
>>> lda = LatentDirichletAllocation(n_topics=3).fit(X)
>>> lda.components_
array([[ 0.91847978,  2.43240154,  2.14285975,  1.10793777],
       [ 2.01874248,  0.92082054,  3.05363433,  3.58789672],
       [ 0.49890124,  0.50614032,  0.47761483,  0.51191327]])
>>> lda.components_.sum(axis=1)
array([ 6.60167884,  9.58109407,  1.99456967])

Looks like it's just not normalized, since normalizing in-place doesn't change transform outputs:

>>> d_before = lda.transform(X)
>>> lda.components_ /= lda.components_.sum(axis=1)[:, np.newaxis]
>>> d_after = lda.transform(X)
>>> norm(d_before - d_after)

But I don't know if a consequent partial_fit will still work.

Follow-up to #6320.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with LatentDirichletAllocation and inspect how components_, transform, and partial_fit are implemented. Check whether normalization is an intended invariant and whether changing components_ affects subsequent partial_fit calls. Done should resolve the normalization behavior while preserving correct transform results and include verification for the partial-fit case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.