deepseek-ai / deepseek-ai/DeepSeek-Coder
使用vllm加速inference后输出容易不符合格式要求
Open
- Dominant language
- Python
- Stars
- 24.3k
- Forks
- 2.9k
- PR merge metrics
- No merged PRs in 30d
Description
我在prompt中要求模型仅输出以def execute_command为开头的python函数,在原生的模型中测试没有问题,但是在利用vllm加速之后,容易输出类似如图
不符合格式要求的输出。
以及也有可能输出对函数为什么要这么写的解释,但我不需要这些额外的输出。
请问有什么办法抑制这样的输出,使其符合我的格式要求吗?
Contributor guide
No contributing guide indexed for this repository
Research direction
No source file, test, or entry point is identified in the issue. First reproduce the same prompt with native inference and vLLM, then compare the generated output and available generation settings; done means establishing a documented or tested way to keep the response limited to the requested Python function.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100