MaartenGr / MaartenGr/BERTopic
Facing problem when loading BERTopic Online Model
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.8k
- Forks
- 920
- Avg merge
- 22h 24m
- Merged PRs (30d)
- 5
Description
Hi @MaartenGr ,
**I am facing the below issue when, I try to load an Online BERTopic model from my disk.
Issue --> 'In order to use `.partial_fit`, the cluster model should have a `.partial_fit` function.'
I tried to debug the code flow where I found that in BERTopic.load() we have a function call to _create_model_from_files() which loads the saved model.
The logic goes like this
```
empty_dimensionality_model = BaseDimensionalityReduction()
empty_cluster_model = BaseCluster()
# Fit BERTopic without actually performing any clustering
topic_model = BERTopic(
embedding_model=embedding_model,
umap_model=empty_dimensionality_model,
hdbscan_model=empty_cluster_model,
**params
)
```
Here, for Online Topic Models we use River.DBSTREAM as cluster model. But when we save the model using BERTopic.save() [in safetensors format] we are not storing the cluster model data.... Because of that, While loading back we face this issue..
Is there any quick fix for this issue??**
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with BERTopic.load() and _create_model_from_files(), then compare them with BERTopic.save() using the safetensors format. Trace how the River.DBSTREAM cluster model is handled during saving and loading; done means an online model can be loaded and used with .partial_fit without the missing-method error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100