[Optimization] Setup slim dependency install for controller and openai
@fozziethebeat is already working on this.
Since Aug 2, 2023.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
Right now, in order to install and deploy a server with *only* the fastchat controller or the fastchat openai shim, you have to install torch and a suite of other fairly large dependencies. The two servers don't actually depend on any heavy dependencies and in theory could be setup and served with much fewer dependencies.
This is to outline a few changes so that users can setup faster installs for CPU only controller and api servers:
* [ ] Move `get_conversation_template` out of `fastchat.model.model_adapter` so that it can be imported without triggering any torch dependencies.
* [ ] Setup a `minimal` install target with only the hard requirements for the server
* [ ] Write docs on how to use the separate installs
* [ ] Write docs and example configs for building slim docker images (and verify they're smaller)
Assuming this is done right it'll save a *lot* on docker image size and setup time for the servers.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.