joshsoftware / joshsoftware/chatbot.ai

Clustering Embeddings based on Orgnization

Open
#30 1 comment 0 reactions 1 assignee Claimed by @Selectus2 View on GitHub
enhancement
Dominant language
Python
Stars
0
Forks
2
PR merge metrics
No merged PRs in 30d

Description

### Currently State
We are storing all the embeddings in the same table which in result resulting into chatbot answering out of context queries too.

For example we are in interaction about company 1 and suddenly if I asked something about company 2 its answering about that also as the embeddings are getting fetched as both companies have some similar words in there website.

### Posible solutions

- DB level options
1. Using different hyper tables for different organisation.
2. Use DB views to isolate the data

- Improve Embedding generation
1. Provide uniq seed value for a specific org to get higher distance between embeddings of different orgs
2. Use BM25 Ranking Algorithm
3. Use TF-IDF encoding along with embedding to enhance the similarity search accuracy.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.