MaartenGr / MaartenGr/BERTopic

Facing problem when loading BERTopic Online Model

Open
#1,703 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.8k
Forks
920
Avg merge
22h 24m
Merged PRs (30d)
5

Description

Hi @MaartenGr ,

**I am facing the below issue when, I try to load an Online BERTopic model from my disk.

Issue --> 'In order to use `.partial_fit`, the cluster model should have a `.partial_fit` function.'

I tried to debug the code flow where I found that in BERTopic.load() we have a function call to _create_model_from_files() which loads the saved model.
The logic goes like this
```
empty_dimensionality_model = BaseDimensionalityReduction()
empty_cluster_model = BaseCluster()

# Fit BERTopic without actually performing any clustering
topic_model = BERTopic(
embedding_model=embedding_model,
umap_model=empty_dimensionality_model,
hdbscan_model=empty_cluster_model,
**params
)
```
Here, for Online Topic Models we use River.DBSTREAM as cluster model. But when we save the model using BERTopic.save() [in safetensors format] we are not storing the cluster model data.... Because of that, While loading back we face this issue..

Is there any quick fix for this issue??**

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with BERTopic.load() and _create_model_from_files(), then compare them with BERTopic.save() using the safetensors format. Trace how the River.DBSTREAM cluster model is handled during saving and loading; done means an online model can be loaded and used with .partial_fit without the missing-method error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.