MaartenGr / MaartenGr/BERTopic

Langchain representation model creates the same representations for all topics (instead of different representations for each topic)

Open
#2,325 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
7.8k
Forks
920
Avg merge
22h 24m
Merged PRs (30d)
5

Description

### Have you searched existing issues? 🔎

- [x] I have searched and found no existing issues

### Desribe the bug

I'm currently working with the Langchain representation model for a project where I need to generate distinct representations for various topics. However, I've encountered an issue where the model produces the same representation for every topic.

```
class LangchainRepresentation(BaseRepresentation):
def __init__(
self,
chain,
prompt: PromptTemplate,
nr_docs: int = 4,
diversity: float = None,
doc_length: int = None,
tokenizer: Union[str, Callable] = None,
chain_config=None,
):
self.chain = chain
self.prompt = PromptTemplate.from_template(prompt)
self.default_prompt_ = DEFAULT_PROMPT
self.chain_config = chain_config
self.nr_docs = nr_docs
self.diversity = diversity
self.doc_length = doc_length
self.tokenizer = tokenizer


def extract_topics(
self,
topic_model,
documents: pd.DataFrame,
c_tf_idf: csr_matrix,
topics: Mapping[str, List[Tuple[str, float]]],
) -> Mapping[str, List[Tuple[str, int]]]:
"""Extract topics.

Arguments:
topic_model: A BERTopic model
documents: All input documents
c_tf_idf: The topic c-TF-IDF representation
topics: The candidate topics as calculated with c-TF-IDF

Returns:
updated_topics: Updated topic representations
"""

# Extract the top 4 representative documents per topic
repr_docs_mappings, _, _, _ = topic_model._extract_representative_docs(
c_tf_idf=c_tf_idf,
documents=documents,
topics=topics,
nr_samples=500,
nr_repr_docs=self.nr_docs,
diversity=self.diversity,
)
# Generate label using langchain's batch functionality, removed truncate documents
chain_docs: List[List[Document]] = [
[
Document(doc)
for doc in docs
]
for docs in repr_docs_mappings.values()
]

# `self.chain` must take `input_documents` and `question` as input keys
# Use a custom prompt that leverages keywords, using the tag: [KEYWORDS]

if "keywords" in set(self.prompt.input_variables):
keywords_list = []
for topic in topics:
keywords = list(zip(*topics[topic]))[0]
keywords_list.append(','.join(keywords))

inputs = [{"input_document": docs, "keywords": keywords} for docs, prompt in zip(chain_docs, keywords_list)]

else:
inputs = [{"input_document": docs,} for docs in chain_docs]

# `self.chain` must return a dict with an `output_text` key
# same output key as the `StuffDocumentsChain` returned by `load_qa_chain`
chain = self.prompt | self.chain
outputs = chain.batch(inputs=inputs,)
labels = [output.content.strip() for output in outputs]
updated_topics = {
topic: [(label, 1)] for topic,label in zip(repr_docs_mappings.keys(), labels)
}

print(f"updated_topics: {updated_topics}")

return updated_topics

langchain_representation_model = LangchainRepresentation(
chain=llm, nr_docs=10,
prompt="""
#Keywords#
{keywords}

#Task#
Based on the provided information filed to a company, derive a label that encapsulates the essence of the information .

#Output Requirement#
- The label should be an impactful phrase that represents the ##derived Summary##.
- The label should also be short and precise.
- Only output the label.
""")

mmr = MaximalMarginalRelevance(diversity=0.7)

topic_model = BERTopic(
min_topic_size=10,
zeroshot_min_similarity=0.7,
ctfidf_model=ClassTfidfTransformer(bm25_weighting=True,reduce_frequent_words=True),
umap_model=umap_model,
vectorizer_model=CountVectorizer(ngram_range=(1,3), stop_words="english"),
representation_model=[mmr, langchain_representation_model],
verbose=True)
```

### Reproduction

```python
from bertopic import BERTopic

```

### BERTopic Version

0.16.4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with LangchainRepresentation.extract_topics and the BERTopic representation_model configuration shown in the report. Inspect the representative-document mappings, generated batch inputs, and returned outputs for multiple topics. Done means the representation model produces distinct topic representations where the input topics require them.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.