MaartenGr / MaartenGr/BERTopic

How to adjust parameters for plotting topic over time like real time distribution❓

Open
#555 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.8k
Forks
920
Avg merge
22h 24m
Merged PRs (30d)
5

Description

Dear MaartenGr:
I think u did a great work in Bertopic, especially for the convenience and visualization. I found an interesting thing that when plotting topics over time, the distribution of years are not sound like real-time. For example, I selected short text files which range from 2000-to 2021. I change the 'global_tuning' and 'evolution_tuning' parameters from True to False, But the topic over time distribution always missing members in 2021. The text files time distribution is nearly average. I sincerely need u help. The followings are my code.

`from bertopic import BERTopic`
`from umap import UMAP`

`from sklearn.feature_extraction.text import CountVectorizer`
`vectorizer_model = CountVectorizer(stop_words="english")`

`umap_model = UMAP(n_neighbors=15, n_components=5, `
`min_dist=0.0, metric='cosine', random_state=42)`
`topic_model = BERTopic(umap_model=umap_model,vectorizer_model=vectorizer_model,top_n_words=30, min_topic_size=35, calculate_probabilities=True, verbose=True)`
`topics, probs = topic_model.fit_transform(tweets)`

`topics_over_time = topic_model.topics_over_time(docs=tweets, `
`topics=topics, `
`timestamps=timestamps, `
`global_tuning=True, `
`evolution_tuning=False, `
`nr_bins=10)`

![image](https://user-images.githubusercontent.com/13728918/171828039-6b725be0-78cc-45cf-ae4c-2055291a0467.png)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with BERTopic's topics_over_time call using the provided tweets, topics, timestamps, global_tuning, evolution_tuning, and nr_bins settings; compare the input timestamps with the resulting time bins, especially 2021. Done means determining why 2021 entries are missing and confirming the resulting topic-over-time distribution matches the supplied timestamps.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, scikit-learn
Domain
data-visualization, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.