huggingface / huggingface/text-clustering

Example in README.md isn't working

Open
#11 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
608
Forks
48
PR merge metrics
No merged PRs in 30d

Description

Kernel crashes upon running the code; repeated attempts yield the same error

```
from src.text_clustering import ClusterClassifier
from datasets import load_dataset

SAMPLE = 100_000

texts = load_dataset("HuggingFaceTB/cosmopedia-100k", split="train").select(range(SAMPLE))["text"]

cc = ClusterClassifier(embed_device="mps")

# run the pipeline:
embs, labels, summaries = cc.fit(texts)

# show the results
cc.show()

# save
cc.save("./cc_100k")
```

```
INFO:datasets:PyTorch version 2.2.2 available.
INFO:sentence_transformers.SentenceTransformer:Load pretrained SentenceTransformer: all-MiniLM-L6-v2
The Kernel crashed while executing code in the current cell or a previous cell.
Please review the code in the cell(s) to identify a possible cause of the failure.
Click [here](https://aka.ms/vscodeJupyterKernelCrash) for more info.
View Jupyter [log](command:jupyter.viewOutput) for further details.
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the README.md example and reproduce it in the Jupyter environment described by the VS Code kernel-crash message. Check the Jupyter output logs and the ClusterClassifier.fit entry point to identify why the 100,000-text pipeline crashes; done means the example completes without crashing and reaches the result display or save step.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, jupyter-notebook, python, pytorch
Domain
documentation, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.