Require parameters for llm drift evals
Open
- Dominant language
- Rust
- Stars
- 13
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
When implementing `LLMetric` as part of `LLMDriftProfile` logic, we'll need to make sure that we add logic to check for bound parameters in evaluation prompts. It should be expected that a combination of `input` or `output` parameters are provided in the prompt.
This is needed because when the server is running LLM evals with the user-defined workflow, it will inject the input and output into the prompts before calling and returning a metric.
Contributor guide
Assessment
This issue has not been assessed yet.