Is it safe and faster to use multiprocess to call response = openai.ChatCompletion.create()?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
With RESTful API Server launched in the background, I try to use multiprocess to call response = openai.ChatCompletion.create().
I found that:
1, the Server sequentially processes my requests one by one, so with only one server it is not faster to use multiprocess to do request. Instead, I have to launch many Servers to gain speed improvement.
2, the responses from multiprocess are reliable, and nothing missed, so with only one server it is safe to handle requests from different people.
My question is:
my two points are correct?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file, test, or server entry point is named; start by examining the RESTful API server and the behavior of openai.ChatCompletion.create() under multiprocessing. Done would require a maintainer-confirmed answer about request serialization and safety, supported by a reproducible benchmark or test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100