microsoft / microsoft/LLMLingua
Which version of openai should be installed to reproduce gsm8k with llmlingua?
Open
@iofu728 is already working on this.
Since Nov 20, 2023.
question
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 428
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 1
Description
I installed the version 0.27.4 for runing the code examples/CoT.ipynb
some error raised when running the following line
request_data = {
"prompt": prompt,
"max_tokens": 400,
"temperature": 0,
"top_p": 1,
"n": 1,
"stream": False,
"stop": "\n\n",
}
response = openai.Completion.create(
model="gpt-3.5-turbo-0301",
**request_data,
)
print(json.dumps(response, indent=4))
openai.error.InvalidRequestErrorpython-BaseException
: This is a chat model and not supported in the v1/completions endpoint. Did you mean to use v1/chat/completions?
I update openai to version 0.28.1, the error also exists. Updating newer version doesn't work.
So I change the code according to the error. It seems gpt-3.5-torbo-0301 only used for ChatCompletion.
request_data = {
"messages": [{"role": "system", "content": ""}, {"role": "user", "content": prompt}],
"max_tokens": 400,
"temperature": 0,
"top_p": 1,
"n": 1,
"stream": False,
"stop": "\n\n",
}
response = openai.ChatCompletion.create(
model="gpt-3.5-turbo-0301",
**request_data,
)
The finally output is 0.439, far from 0.78+.
num_q 1319 correct 579 ratio 0.4390
Is there any suggestion for me?
- Should I use older openai or newer openai?
- Older openai may not support newer models, so I prefer to use newer openai (for more widely testing) to reproduce the result. But the result of
gpt-3.5-torbo-0301seems not good.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.