huggingface / huggingface/lighteval

[FT] Pass reasoning effort to `litellm` endpoint.

Open
#918 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
555
Avg merge
1d 6h
Merged PRs (30d)
1

Description

## Issue encountered
There is currently no way to pass the `reasoning_effort` parameter to `litellm.completion(...)`. Without being able to turn on reasoning there is no way to do most flagship models justice. This might be a more general issue with other entrypoints too, am not familiar. LiteLLM supports optional literal values `minimal`, `low`, `medium`, `high`.

## Solution/Feature
Would like to be able to specify something like

```python
model_config = LiteLLMModelConfig(
model_name=model_name,
api_key=api_key,
generation_parameters=GenerationParameters(
reasoning_effort="high"
)
)
```

## Possible alternatives
Perhaps adding it through `GenerationParameters` doesn't make sense, in which case it could be added to `LiteLLMModelConfig` directly.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.