awslabs / awslabs/llmeter

Unpinned s3fs pulls in an ancient and slow version

Open Beginner friendly
#77 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
41
Forks
5
Avg merge
2m
Merged PRs (30d)
1

Description

Installing latest LLMeter on a fresh environment pulls in `s3fs==0.4.2` ([from 2020](https://github.com/fsspec/s3fs/releases/tag/0.4.2)) - and using S3 features raises the following warning:

```
UserWarning: Your installed version of s3fs is very old and known to cause
severe performance issues, see also https://github.com/dask/dask/issues/10276

To fix, you should specify a lower version bound on s3fs, or
update the current installation.

warnings.warn(s3_msg)
```

As mentioned in the cited thread, it seems like this is caused how dependency resolvers handle the vastly different release frequencies of s3fs, aiobotocore, and botocore/boto3.

Maybe LLMeter should pin s3fs at some reasonable compromise minimal version (e.g. `>=2023`?) to avoid the issue?

Contributor guide

Open the contributing guide

Research direction

Start by locating LLMeter's dependency declarations and reproduce a fresh installation of the latest release. Check how the resolver selects s3fs alongside aiobotocore and botocore/boto3, then choose and document a lower bound that avoids the reported old-version warning. Confirm that using S3 features no longer emits the warning.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
cloud, tooling
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.