aws-samples / aws-samples/foundation-model-benchmarking-tool
Loosen dependencies for more flexible installation
- Dominant language
- Jupyter Notebook
- Stars
- 256
- Forks
- 44
- PR merge metrics
- No merged PRs in 30d
Description
As discussed [here on StackOverflow](https://stackoverflow.com/a/44938662):
1. **Applications** should generally lock dependencies to exact versions, for reliable deployment
2. **Libraries** should generally support broad dependency version ranges where practical, to accommodate installing them on a range of environments and using them in a range of downstream applications
I suggest that `poetry.lock` accomplishes (1) for users wishing to download fmbench from source, but users installing fmbench from PyPI fall in the camp of (2), and would like fmbench to play nicely with whatever other dependencies might be in their environment.
For stable libraries that follow semver, it seems like we should be able to trust [caret requirements](https://python-poetry.org/docs/dependency-specification/#caret-requirements)? For e.g. specifically I would think something like the below (which I haven't fully tested):
- ipywidgets `8.1.1` -> `^8.0.0` (unless we care about [specific bug fixes they released](https://github.com/jupyter-widgets/ipywidgets/releases)?)
- transformers `4.36.2` -> `^4.36.2` (Idk which potentially new models you're consuming that might prevent downgrade)
- pandas `2.1.4` -> `^2.0.0`(even this is only like [a year old](https://pandas.pydata.org/pandas-docs/stable/whatsnew/index.html#version-2-0)?)
- datasets `2.16.1` -> `^2.14.0` (2.14.0 has an [important caching change](https://github.com/huggingface/datasets/releases?page=2))
- sagemaker `2.220.0` -> `^2.119.0` (Current SMStudio / SageMaker Distribution v1.8 version)
- litellm `1.35.8` -> `^1.35.8` (idk how far back we could push this?)
- plotly `5.22.0` -> `^5.15.0` (before which there were [compatibility issues with Pandas 2.0](https://github.com/plotly/plotly.py/releases))
For unstable libraries (`seaborn`, `tomark`, `kaleido`), maybe we could at least use [tilde requirements](https://python-poetry.org/docs/dependency-specification/#tilde-requirements) to allow patch versions?
Contributor guide
Assessment
This issue has not been assessed yet.