Document diverged OpenAI API
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
The existence of the diverged OpenAI API appears to be undocumented at the moment, but has some useful and requested features (e.g. `repetition_penalty` #1914).
As far as I can tell, it appears these endpoints can be used with the OpenAI client, and additional parameters passed with the `extra_body` argument, e.g.:
```python
# ...
openai.base_url = "http://localhost:8000/api/v1/"
# ...
completion = openai.chat.completions.create(
model=model,
messages=[{"role": "user", "content": "Hello! What is your name?"}],
extra_body={
"repetition_penalty": 1.10
}
)
```
(this is also useful for `top_k` which is not a supported argument for the OpenAI client but is supported in both APIs in FastChat)
It would be good to at least document that this API exists and confirm whether the divergences actually impact using the OpenAI client.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating FastChat's OpenAI-compatible API documentation and serving entry points, then compare the documented endpoints with the OpenAI client example in the issue. Document that the diverged API exists and clarify whether extra_body supports parameters such as repetition_penalty and top_k.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100