[Feature] Support for Mistral
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 748
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 54
Description
### Motivation
Mistral 7b is currently the best LLM <10b (and maybe even <30b), as per latest HELM evaluation (https://crfm.stanford.edu/helm/v0.4.0/#/leaderboard) and also scores very high on Open LLM Leaderboards on HuggingFace (https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).
In my personal experience, it is by far the easiest model to fine-tune to great results - it is small and the sliding window mechanism means that it is easy to fine-tune on 8k/16k context window without OOM.
Therefore, I would really appreciate it if support would be added for it. There are also rumors that Mistral will soon release another LLM as open source, and when that happens it will likely be the new SOTA #1, so I think having day-0 support would be great for lmdeploy.
Thank you very much for your amazing library!
### Related resources
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.