huggingface / huggingface/lighteval
[FT] Pass reasoning effort to `litellm` endpoint.
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 555
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 1
Description
## Issue encountered
There is currently no way to pass the `reasoning_effort` parameter to `litellm.completion(...)`. Without being able to turn on reasoning there is no way to do most flagship models justice. This might be a more general issue with other entrypoints too, am not familiar. LiteLLM supports optional literal values `minimal`, `low`, `medium`, `high`.
## Solution/Feature
Would like to be able to specify something like
```python
model_config = LiteLLMModelConfig(
model_name=model_name,
api_key=api_key,
generation_parameters=GenerationParameters(
reasoning_effort="high"
)
)
```
## Possible alternatives
Perhaps adding it through `GenerationParameters` doesn't make sense, in which case it could be added to `LiteLLMModelConfig` directly.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.