Add llm-embedding-eval to plugin directory
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 12.5k
- Forks
- 998
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 10
Description
Add llm-embedding-eval to plugin directory
LLM Embedding Eval
Hi Simon! I've created a plugin that will compare two embeddings and determine their similarity/relevance based on various metrics. It also supports SemScore as proposed in a publication(https://arxiv.org/abs/2401.17072).
The basic idea is to evaluate similarity between two embeddings using various similarity metrics on-the-fly. It supports both binary embedding files and SQLite DB files (.db extension).
Pls check out the repo here (https://github.com/ajayarunachalam/llm-embedding-eval)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the plugin directory and reviewing how existing plugins are listed. Add LLM Embedding Eval with the repository link and description from the issue, then verify that the directory renders the new entry correctly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sqlite
- Domain
- cli
- Issue type
- Feature
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100