MaartenGr / MaartenGr/BERTopic

Memory Management: Allow None embedding_model with pre-computed embeddings

Open
#2,271 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.8k
Forks
920
Avg merge
22h 24m
Merged PRs (30d)
5

Description

First, I want to preface that I'm not an expert with BERTopic's internals, so I apologize if I'm misunderstanding something fundamental.
I'm running into an issue where I have limited VRAM and want to use BERTopic with an LLM for topic labeling. My workflow involves:

- Pre-computing embeddings using a large embedding model
- Saving these embeddings to disk
- Loading them back for topic modeling

The challenge I'm facing is that when I try to initialize BERTopic with embedding_model=None while providing pre-computed embeddings, I get an error. This means I have to keep the large embedding model in memory even though I've already generated the embeddings.

Here's a minimal example of my current workaround using a dummy embedder:

```
class DummyEmbedder:
def embed_documents(self, documents, verbose=False):
return embeddings # Using precomputed embeddings

def embed_words(self, words, verbose=False):
return embeddings

dummy_embedder = DummyEmbedder()

topic_model = BERTopic(
embedding_model=dummy_embedder,
umap_model=reduced_e,
hdbscan_model=hdbscan_model,
vectorizer_model=vectorizer_model,
ctfidf_model=ctfidf_model,
representation_model=representation_models,
)
topics, probs = topic_model.fit_transform(data, embeddings=embeddings)
```

Question
- Is there a better way to handle this scenario?
- Would it make sense to allow embedding_model=None when pre-computed embeddings are provided?
- Is my dummy embedder approach safe to use, or could it cause issues I'm not aware of?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the minimal BERTopic example and inspect how BERTopic initialization and fit_transform handle embedding_model and pre-computed embeddings. Done should mean pre-computed embeddings can be used with embedding_model=None without retaining the original embedding model, while existing embedding workflows remain safe.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.