Embedding metrics
- Dominant language
- Rust
- Stars
- 40
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
LLM as a judge is important, but it's also imperative to provide some form of statistical insight into how data is changing over time when it comes to inputs and outputs.
Use case:
A user would like to access changes in the distributions of the their input and outputs over time
Approach:
- Generate embeddings for input and output
- Average the embedding? - tbd, research this
- Accumulate results over time:
- similarities
- Skew
- Mean,
- median
- mode
- distribution plots, etc.
Contributor guide
Research direction
No file, test, or entry point is named. Start by researching how input and output embeddings should be generated and aggregated over time, then define the required similarity statistics, distribution measures, and plots; done criteria are not specified in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- analytics, machine-learning, observability-sre
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100