MaartenGr / MaartenGr/BERTopic

Updating and Pushing a BERTopic Model with New Documents to Hugging Face Hub still shows old no of training document

Open
#2,071 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
7.8k
Forks
920
Avg merge
22h 24m
Merged PRs (30d)
5

Description

### Have you searched existing issues? 🔎

- [X] I have searched and found no existing issues

### Desribe the bug

I have been using BERTopic for topic modelling and recently needed to update my existing BERTopic model with new documents. I want to push the updated model to the Hugging Face Hub, ensuring that it reflects the new number of documents and topics.

Here’s what I’ve done so far:

- Loaded my existing BERTopic model:
- Added new documents and their embeddings:
- Updated the model with new documents:

```
`new_topics, new_probs = topic_model.transform(lemmatized_docs, embeddings)`
```

- Saved the updated model using safetensors:
- Pushed the updated model to Hugging Face Hub:

Despite following these steps, I still see the old number of training documents in the repository on the Hugging Face Hub. How can I ensure that the updated model reflects the new number of training and topics?

Any help or guidance on this would be greatly appreciated!

### Reproduction

```python
from bertopic import BERTopic

# Load your existing BERTopic model
topic_model= BERTopic.load("shantanudave/BERTopic_ArXiv",embedding_model="sentence-transformers/all-MiniLM-L6-v2")

new_topics, new_probs = topic_model.transform(lemmatized_docs, embeddings)

new_model_name = "BERTopic_v2"

# Save the updated model locally using safetensors

embedding_model = "sentence-transformers/all-MiniLM-L6-v2"
topic_model.save(new_model_name, serialization="safetensors", save_ctfidf=True, save_embedding_model=embedding_model)

from huggingface_hub import login

# Authenticate with Hugging Face
login(token="your_hugging_face_token")

# Push the updated model to Hugging Face Hub
topic_model.push_to_hf_hub(
repo_id=f"shantanudave/{new_model_name}",
serialization="safetensors",
save_ctfidf=True,
save_embedding_model=embedding_model
)

```

### BERTopic Version

pip install -U bertopic

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reproduction using BERTopic.load, transform, save, and push_to_hf_hub. Inspect how model metadata is written during saving and uploading, then verify whether the Hub artifact reflects the expected training-document and topic counts. Done means the behavior is explained and, if it is a bug, covered by a reproducible regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
cloud, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.