lm-sys / lm-sys/FastChat

poor performance of httpx.AsyncClient in openai_api_server.py

Open
#2,256 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
39.5k
Forks
4.8k
PR merge metrics
No merged PRs in 30d

Description

Hi,

While I stress "/v1/completions" interface provided by fastchat openai api server, I find its concurrency performance is very poor.
I add some debug message in process function [create_completion](https://github.com/lm-sys/FastChat/blob/main/fastchat/serve/openai_api_server.py#L458), and find when 100 request is sent in the mean time, the request would be processed nearly sequential in this function, which made response very slow.

I try change httpx.AsyncClient post method into aiohttp, and get the performance boost in this way.
So for the performance, any idea for why httpx.AsyncClient works poor in the high concurrency mode?

If needed, I could push my PR to switch httpx.AsyncClient into aiohttp.

Thx

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in fastchat/serve/openai_api_server.py at create_completion, then reproduce the reported load against the /v1/completions endpoint with concurrent requests. Compare the httpx.AsyncClient behavior with the reported aiohttp result and identify whether the client choice explains the near-sequential processing. Done means the cause is established and a verified performance fix or clear resolution is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.