MaartenGr / MaartenGr/BERTopic

How to do Hierarchical Topic Modeling on Merged Model?

Open
#2,152 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
7.8k
Forks
920
Avg merge
22h 24m
Merged PRs (30d)
5

Description

### Have you searched existing issues? 🔎

- [X] I have searched and found no existing issues

### Desribe the bug

Hi @MaartenGr
I am trying to create visualization of hierarchical topic modeling on two topic models merged using .merge_models.

```
hierarchical_topics_merged = merged_model.hierarchical_topics(docs_1+docs_2)
```

It produces the following error:

```
2024-09-16 09:47:47,878 - BERTopic - WARNING: No c-TF-IDF matrix was found despite it is supposed to be used (`use_ctfidf` is True). Defaulting to semantic embeddings.
---------------------------------------------------------------------------
NotFittedError Traceback (most recent call last)
[](https://localhost:8080/#) in ()
----> 1 hierarchical_topics_merged = merged_model.hierarchical_topics(docs_3)

2 frames
[/usr/local/lib/python3.10/dist-packages/sklearn/feature_extraction/text.py](https://localhost:8080/#) in _check_vocabulary(self)
506 self._validate_vocabulary()
507 if not self.fixed_vocabulary_:
--> 508 raise NotFittedError("Vocabulary not fitted or provided")
509
510 if len(self.vocabulary_) == 0:

NotFittedError: Vocabulary not fitted or provided
```

How do I visualize merged models?

Thanks!

### BERTopic Version

v0.16.3

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the call to merged_model.hierarchical_topics(docs_1+docs_2) after merge_models, then inspect the c-TF-IDF handling and the NotFittedError from sklearn's text feature extraction. Check the hierarchical_topics and merge_models entry points. Done means either merged models can be visualized successfully or the supported workflow and limitation are documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, scikit-learn
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.