[FEATURE] Truncation support for non vllm_server providers
Offen
enhancement
- Vorherrschende Sprache
- Python
- Sterne
- 70
- Forks
- 21
- Ø Merge
- 2 T. 11 Std.
- Gemergte PRs (30 T.)
- 41
Beschreibung
## Feature Description
Add in support for the sampling_params variables truncation_side and truncate_prompt_tokens to the vllm, huggingface, litellm, and olmo_core providers to match the vllm_server implementation (https://github.com/allenai/olmo-eval/pull/208).
## Use Case
Currently, any setting of truncation_side or truncate_prompt_tokens for a non-vllm_server provider generates a warning. This limits providers that can be used for the safety judge, and any other length-limited model that is run
Beitragsleitfaden
Für dieses Repository ist kein Beitragsleitfaden indexiert
Bewertung
Dieses Issue wurde noch nicht bewertet.