deepseek-ai / deepseek-ai/DeepSeek-Coder
使用fim后human eval分数很低?
- Dominant language
- Python
- Stars
- 24.3k
- Forks
- 2.9k
- PR merge metrics
- No merged PRs in 30d
Description
不使用fim,跑出来的分数跟官方给的比较接近,但是使用fim后分数很低,低了8个点,以下是我使用的fim格式,请问有啥问题吗?
```
def get_full_prompt(model_name, prompt_prefix, prompt, suffix="\n"):
"""
get_full_prompt 通过匹配模型名,使用不同的FIM格式来准备prompt
"""
output = prompt
if "star-coder" in model_name.lower():
...
elif "deep-seek" in model_name.lower() or "deepseek" in model_name.lower():
template = Template("<|fim▁begin|>${preprefix}${prefix}<|fim▁hole|>${suffix}<|fim▁end|>")
output = template.substitute(preprefix=prompt_prefix, prefix=prompt, suffix=suffix)
else:
if prompt_prefix:
output = prompt_prefix + output
return output
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.