FormulaMonks / FormulaMonks/kurt

Add parameter to set PRNG seed

Open
#47 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
TypeScript
Stars
4
Forks
0
PR merge metrics
No merged PRs in 30d

Description

For LLM providers that allow setting the PRNG seed for sampling determinism reasons, Kurt should provide a way to do this.

Current expectations of support across LLM providers:
- OpenAI has this as a documented feature
- VertexAI Gemini doesn't appear to have it, but it may be an [undocumented feature](https://github.com/BerriAI/litellm/issues/2973#issuecomment-2065344363)
- Anthropic Claude doesn't appear to have it.
- Local LLMs should be able to easily support it.

For LLMs which do not support this feature, if the optional `seed` parameter is given by the user, Kurt should throw a `KurtFeatureIncompatibleError` (an error class which doesn't yet exist, but I've been thinking about for some time). We should also include a row for this in the #28 to highlight which LLM API providers have and don't have this feature.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.