microsoft / microsoft/LLMLingua

Which version of openai should be installed to reproduce gsm8k with llmlingua?

Open
#17 3 comments 0 reactions 1 assignee View on GitHub

@iofu728 is already working on this.

Since Nov 20, 2023.

question
Dominant language
Python
Stars
6.7k
Forks
428
Avg merge
2d 4h
Merged PRs (30d)
1

Description

I installed the version 0.27.4 for runing the code examples/CoT.ipynb
some error raised when running the following line

request_data = {
    "prompt": prompt,
    "max_tokens": 400,
    "temperature": 0,
    "top_p": 1,
    "n": 1,
    "stream": False,
    "stop": "\n\n",
}
response = openai.Completion.create(
    model="gpt-3.5-turbo-0301",
    **request_data,
)
print(json.dumps(response, indent=4))
openai.error.InvalidRequestErrorpython-BaseException
: This is a chat model and not supported in the v1/completions endpoint. Did you mean to use v1/chat/completions?

I update openai to version 0.28.1, the error also exists. Updating newer version doesn't work.
So I change the code according to the error. It seems gpt-3.5-torbo-0301 only used for ChatCompletion.

request_data = {
    "messages": [{"role": "system", "content": ""}, {"role": "user", "content": prompt}],
    "max_tokens": 400,
    "temperature": 0,
    "top_p": 1,
    "n": 1,
    "stream": False,
    "stop": "\n\n",
}
response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo-0301",
    **request_data,
)

The finally output is 0.439, far from 0.78+.

num_q 1319 correct 579 ratio 0.4390

Is there any suggestion for me?

  1. Should I use older openai or newer openai?
  2. Older openai may not support newer models, so I prefer to use newer openai (for more widely testing) to reproduce the result. But the result of gpt-3.5-torbo-0301 seems not good.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.