Enabling Upstage's Solar-Open-100B
Open
vllm
- Dominant language
- C++
- Stars
- 529
- Forks
- 80
- Avg merge
- 9h 7m
- Merged PRs (30d)
- 38
Description
Hardware:
- 4x B60 with Xeon CPU
Model
- Upstage's Solar-Open-100B
As we try enable/run [the model](https://huggingface.co/upstage/Solar-Open-100B), it asks to use image `upstage/vllm-solar-open:latest` or install from source on https://github.com/UpstageAI/vllm.git@v0.12.0-solar-open. Seeing from its branch name v0.12.0, its vllm repo was updated on top of v0.12.0.
The model is one of the three finalists of Korea national AI models, and we'd like to enable the model proactively on B60/B70. Wanted to let the team know that XPU llm-scaler image with vLLM v0.12.0 is very much anticipated, and hopefully the Upstage's model is supported on it without too much effort.
Contributor guide
Assessment
This issue has not been assessed yet.