huggingface / huggingface/lighteval

[FT] LiteLLM concurrency parameters hard-coded

Open
#567 1 comment 0 reactions 0 assignees View on GitHub
feature good first issue help wanted
Dominant language
Python
Stars
2.5k
Forks
555
Avg merge
1d 6h
Merged PRs (30d)
1

Description

I noticed that for LiteLLM the concurrency is hard-coded for an arbitrary number (20). Well, a lot of the request behavior is hard coded, but the concurrency is the one that is probably least likely to work well as a one size fits all.

If everything is going through the YAML file, maybe it can be expanded, although the YAML file is another ball of wax (makes automated runs much harder when you can't just pass in parameters via a call)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.