aws-samples / aws-samples/foundation-model-benchmarking-tool

Loosen dependencies for more flexible installation

Open
#105 1 comment 0 reactions 1 assignee Claimed by @aarora79 View on GitHub
Dominant language
Jupyter Notebook
Stars
256
Forks
44
PR merge metrics
No merged PRs in 30d

Description

As discussed [here on StackOverflow](https://stackoverflow.com/a/44938662):
1. **Applications** should generally lock dependencies to exact versions, for reliable deployment
2. **Libraries** should generally support broad dependency version ranges where practical, to accommodate installing them on a range of environments and using them in a range of downstream applications

I suggest that `poetry.lock` accomplishes (1) for users wishing to download fmbench from source, but users installing fmbench from PyPI fall in the camp of (2), and would like fmbench to play nicely with whatever other dependencies might be in their environment.

For stable libraries that follow semver, it seems like we should be able to trust [caret requirements](https://python-poetry.org/docs/dependency-specification/#caret-requirements)? For e.g. specifically I would think something like the below (which I haven't fully tested):
- ipywidgets `8.1.1` -> `^8.0.0` (unless we care about [specific bug fixes they released](https://github.com/jupyter-widgets/ipywidgets/releases)?)
- transformers `4.36.2` -> `^4.36.2` (Idk which potentially new models you're consuming that might prevent downgrade)
- pandas `2.1.4` -> `^2.0.0`(even this is only like [a year old](https://pandas.pydata.org/pandas-docs/stable/whatsnew/index.html#version-2-0)?)
- datasets `2.16.1` -> `^2.14.0` (2.14.0 has an [important caching change](https://github.com/huggingface/datasets/releases?page=2))
- sagemaker `2.220.0` -> `^2.119.0` (Current SMStudio / SageMaker Distribution v1.8 version)
- litellm `1.35.8` -> `^1.35.8` (idk how far back we could push this?)
- plotly `5.22.0` -> `^5.15.0` (before which there were [compatibility issues with Pandas 2.0](https://github.com/plotly/plotly.py/releases))

For unstable libraries (`seaborn`, `tomark`, `kaleido`), maybe we could at least use [tilde requirements](https://python-poetry.org/docs/dependency-specification/#tilde-requirements) to allow patch versions?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.