MaartenGr / MaartenGr/BERTopic
Updating and Pushing a BERTopic Model with New Documents to Hugging Face Hub still shows old no of training document
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.8k
- Forks
- 920
- Avg merge
- 22h 24m
- Merged PRs (30d)
- 5
Description
### Have you searched existing issues? 🔎
- [X] I have searched and found no existing issues
### Desribe the bug
I have been using BERTopic for topic modelling and recently needed to update my existing BERTopic model with new documents. I want to push the updated model to the Hugging Face Hub, ensuring that it reflects the new number of documents and topics.
Here’s what I’ve done so far:
- Loaded my existing BERTopic model:
- Added new documents and their embeddings:
- Updated the model with new documents:
```
`new_topics, new_probs = topic_model.transform(lemmatized_docs, embeddings)`
```
- Saved the updated model using safetensors:
- Pushed the updated model to Hugging Face Hub:
Despite following these steps, I still see the old number of training documents in the repository on the Hugging Face Hub. How can I ensure that the updated model reflects the new number of training and topics?
Any help or guidance on this would be greatly appreciated!
### Reproduction
```python
from bertopic import BERTopic
# Load your existing BERTopic model
topic_model= BERTopic.load("shantanudave/BERTopic_ArXiv",embedding_model="sentence-transformers/all-MiniLM-L6-v2")
new_topics, new_probs = topic_model.transform(lemmatized_docs, embeddings)
new_model_name = "BERTopic_v2"
# Save the updated model locally using safetensors
embedding_model = "sentence-transformers/all-MiniLM-L6-v2"
topic_model.save(new_model_name, serialization="safetensors", save_ctfidf=True, save_embedding_model=embedding_model)
from huggingface_hub import login
# Authenticate with Hugging Face
login(token="your_hugging_face_token")
# Push the updated model to Hugging Face Hub
topic_model.push_to_hf_hub(
repo_id=f"shantanudave/{new_model_name}",
serialization="safetensors",
save_ctfidf=True,
save_embedding_model=embedding_model
)
```
### BERTopic Version
pip install -U bertopic
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reproduction using BERTopic.load, transform, save, and push_to_hf_hub. Inspect how model metadata is written during saving and uploading, then verify whether the Hub artifact reflects the expected training-document and topic counts. Done means the behavior is explained and, if it is a bug, covered by a reproducible regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python
- Domain
- cloud, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100