JetBrains / JetBrains/koog

Add extra_body to open AI class

Open
#504 3 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Kotlin
Stars
4.6k
Forks
474
Avg merge
58m
Merged PRs (30d)
1

Description

It would be great if you can add extra_body to open AI LLM class. This will make it more useful when having an open source backend server like VLLM or SGLANG. The extra_body allow using more customization that are not available in LLM providers like OPENAI.
for example it can be used to specify grammar.
[SGLANG](https://docs.sglang.ai/backend/structured_outputs.html#EBNF)
[VLLM](https://docs.vllm.ai/en/latest/features/structured_outputs.html#online-serving-openai-api)

Contributor guide

Open the contributing guide

Research direction

Start by locating the Open AI LLM class and tracing how its request options are assembled. Use the linked SGLANG and vLLM structured-output examples to determine how extra_body should be passed, then verify that custom fields such as grammar reach compatible backend servers.

Written by the indexing model from the issue text.

Assessment

Tech stack
kotlin
Domain
ai, backend-api-design
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.