vllm-project / vllm-project/aibrix

jinja2.exceptions.UndefinedError: 'bos_token' is undefined

Open
#589 1 comment 0 reactions 1 assignee Claimed by @scarlet25151 View on GitHub
area/benchmark
Dominant language
Go
Stars
5.1k
Forks
694
Avg merge
1d 19h
Merged PRs (30d)
98

Description

### 🐛 Describe the bug

![Image](https://github.com/user-attachments/assets/f00a5795-5ea3-4341-9491-d95d167ae40e)

### Steps to Reproduce

deploy the models
```
vllm serve Qwen/Qwen2.5-Coder-7B-Instruct --enable-lora --lora-modules model-1=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c model-2=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c model-3=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c model-4=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c --max-lora-rank 64
```

send the request
```
curl -v "http://localhost:8000/v1/chat/completions" \
-H "Content-Type: application/json" \
-d '{
"model": "model-1",
"messages": [{"role": "user", "content": "help me run a python randome generator method"}],
"temperature": 0.7
}'
```

### Expected behavior

_No response_

### Environment

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.