MaartenGr / MaartenGr/BERTopic
Memory Management: Allow None embedding_model with pre-computed embeddings
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.8k
- Forks
- 920
- Avg merge
- 22h 24m
- Merged PRs (30d)
- 5
Description
First, I want to preface that I'm not an expert with BERTopic's internals, so I apologize if I'm misunderstanding something fundamental.
I'm running into an issue where I have limited VRAM and want to use BERTopic with an LLM for topic labeling. My workflow involves:
- Pre-computing embeddings using a large embedding model
- Saving these embeddings to disk
- Loading them back for topic modeling
The challenge I'm facing is that when I try to initialize BERTopic with embedding_model=None while providing pre-computed embeddings, I get an error. This means I have to keep the large embedding model in memory even though I've already generated the embeddings.
Here's a minimal example of my current workaround using a dummy embedder:
```
class DummyEmbedder:
def embed_documents(self, documents, verbose=False):
return embeddings # Using precomputed embeddings
def embed_words(self, words, verbose=False):
return embeddings
dummy_embedder = DummyEmbedder()
topic_model = BERTopic(
embedding_model=dummy_embedder,
umap_model=reduced_e,
hdbscan_model=hdbscan_model,
vectorizer_model=vectorizer_model,
ctfidf_model=ctfidf_model,
representation_model=representation_models,
)
topics, probs = topic_model.fit_transform(data, embeddings=embeddings)
```
Question
- Is there a better way to handle this scenario?
- Would it make sense to allow embedding_model=None when pre-computed embeddings are provided?
- Is my dummy embedder approach safe to use, or could it cause issues I'm not aware of?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the minimal BERTopic example and inspect how BERTopic initialization and fit_transform handle embedding_model and pre-computed embeddings. Done should mean pre-computed embeddings can be used with embedding_model=None without retaining the original embedding model, while existing embedding workflows remain safe.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100