huggingface / huggingface/lighteval

[FT] Allow configuration of `litellm` cache location

Open
#806 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
555
Avg merge
1d 6h
Merged PRs (30d)
1

Description

## Issue encountered

In `lighteval==0.10.0`, the creation of `litellm`'s `Cache`: https://github.com/huggingface/lighteval/blob/v0.10.0/src/lighteval/models/litellm_model.py#L59

Does not allow for configuration of the cache location.

## Solution/Feature

Allow configuration of the cache location via an environment variable like so:

```python
LIGHTEVAL_LITELLM_CACHE_LOCATION = os.environ.get("LIGHTEVAL_LITELLM_CACHE_LOCATION")

litellm.cache = Cache(type="disk", disk_cache_dir=LIGHTEVAL_LITELLM_CACHE_LOCATION)
```

## Possible alternatives

N/a

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.