vllm-project / vllm-project/aibrix
jinja2.exceptions.UndefinedError: 'bos_token' is undefined
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 694
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 98
Description
### 🐛 Describe the bug

### Steps to Reproduce
deploy the models
```
vllm serve Qwen/Qwen2.5-Coder-7B-Instruct --enable-lora --lora-modules model-1=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c model-2=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c model-3=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c model-4=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c --max-lora-rank 64
```
send the request
```
curl -v "http://localhost:8000/v1/chat/completions" \
-H "Content-Type: application/json" \
-d '{
"model": "model-1",
"messages": [{"role": "user", "content": "help me run a python randome generator method"}],
"temperature": 0.7
}'
```
### Expected behavior
_No response_
### Environment
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.