microsoft / microsoft/foundry-dev-tools
frequency_penalty at 0 causes no response content with Phi-3-mini-4k-cpu-int4-rtn-block-32-acc-level-4-onnx
Open
@a1exwang is already working on this.
Since Aug 23, 2024.
bug
- Dominant language
- JavaScript
- Stars
- 2.1k
- Forks
- 260
- Avg merge
- 42m
- Merged PRs (30d)
- 29
Description
Frequency response at 0 causes an issue with no content in the response. > 0 by < 1 cause other weird responses. 1 seems to be the only reliable value and its unclear if its the model or something else.
POST http://127.0.0.1:5272/v1/chat/completions
content-type: application/json
{
"messages": [
{
"role": "user",
"content": "Whats the golden ratio"
}
],
"frequency_penalty": 0,
"model": "Phi-3-mini-4k-cpu-int4-rtn-block-32-acc-level-4-onnx"
}
You will get a response like:
{
"model": null,
"choices": [
{
"delta": {
"role": "assistant",
"content": "",
"name": null,
"tool_call_id": null,
"function_call": null,
"tool_calls": null
},
"message": {
"role": "assistant",
"content": "",
"name": null,
"tool_call_id": null,
"function_call": null,
"tool_calls": null
},
"index": 0,
"finish_reason": "stop",
"finish_details": null,
"logprobs": null
}
],
"usage": null,
"created": 1724095112,
"id": "chat.id.2641",
"system_fingerprint": null,
"object": "chat.completion",
"Successful": true,
"error": null,
"HttpStatusCode": 0,
"HeaderValues": null
}
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.