x-tabdeveloping / x-tabdeveloping/turftopic
Divide Overflow when subdividing groups
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 123
- Forks
- 9
- Avg merge
- 15h 59m
- Merged PRs (30d)
- 1
Description
A division overflow can occur when subdividing groups multiple times.
Example:
from sklearn.datasets import fetch_20newsgroups
from sklearn.feature_extraction.text import CountVectorizer
from turftopic import KeyNMF
def split_groups(groups, n):
if n == 0:
return
for group in groups:
group.divide(5)
split_groups(group, n - 1)
if __name__ == '__main__':
corpus = fetch_20newsgroups(subset='all', remove=('headers', 'footers', 'quotes')).data
corpus = corpus[0:250]
vectorizer = CountVectorizer(min_df=5, max_df=0.8, stop_words='english')
model = KeyNMF(n_components=5, vectorizer=vectorizer)
topic_data = model.prepare_topic_data(corpus)
split_groups(topic_data.hierarchy, 3)
Error:
/.venv1/lib/python3.10/site-packages/turftopic/models/wnmf.py:40: RuntimeWarning:
overflow encountered in divide
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the provided recursive subdivision example with the 20 Newsgroups corpus and inspect turftopic/models/wnmf.py at line 40, where the RuntimeWarning is reported. Trace the repeated group divisions and verify that the example completes without a division-overflow warning.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100