huggingface / huggingface/lighteval
[FT] Allow configuration of `litellm` cache location
Open
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 555
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 1
Description
## Issue encountered
In `lighteval==0.10.0`, the creation of `litellm`'s `Cache`: https://github.com/huggingface/lighteval/blob/v0.10.0/src/lighteval/models/litellm_model.py#L59
Does not allow for configuration of the cache location.
## Solution/Feature
Allow configuration of the cache location via an environment variable like so:
```python
LIGHTEVAL_LITELLM_CACHE_LOCATION = os.environ.get("LIGHTEVAL_LITELLM_CACHE_LOCATION")
litellm.cache = Cache(type="disk", disk_cache_dir=LIGHTEVAL_LITELLM_CACHE_LOCATION)
```
## Possible alternatives
N/a
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.