Add extra_body to open AI class
- Dominant language
- Kotlin
- Stars
- 4.6k
- Forks
- 474
- Avg merge
- 58m
- Merged PRs (30d)
- 1
Description
It would be great if you can add extra_body to open AI LLM class. This will make it more useful when having an open source backend server like VLLM or SGLANG. The extra_body allow using more customization that are not available in LLM providers like OPENAI.
for example it can be used to specify grammar.
[SGLANG](https://docs.sglang.ai/backend/structured_outputs.html#EBNF)
[VLLM](https://docs.vllm.ai/en/latest/features/structured_outputs.html#online-serving-openai-api)
Contributor guide
Research direction
Start by locating the Open AI LLM class and tracing how its request options are assembled. Use the linked SGLANG and vLLM structured-output examples to determine how extra_body should be passed, then verify that custom fields such as grammar reach compatible backend servers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kotlin
- Domain
- ai, backend-api-design
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100