Lightning-AI / Lightning-AI/LitServe
OpenAISpec rejects standard required and named function tool_choice values
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.9k
- Forks
- 304
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 6
Description
## 🐛 Bug
`OpenAISpec` rejects two standard Chat Completions `tool_choice` values during request validation:
- `"required"`
- a named function choice such as `{"type": "function", "function": {"name": "lookup_weather"}}`
The current request model accepts only `"auto"`, `"none"`, and the non-standard compatibility value `"any"`. As a result, clients using either standard form receive HTTP 422 before `LitAPI.decode_request` runs. OpenAI's current [Chat Completions API reference](https://developers.openai.com/api/reference/cli/resources/chat) documents both `"required"` and a named function object in `tool_choice`.
### Reproduction
On current `main` (`7675990`):
```python
from litserve.specs.openai import ChatCompletionRequest
base = {
"model": "lit",
"messages": [{"role": "user", "content": "Use the weather tool"}],
}
ChatCompletionRequest(**base, tool_choice="required")
ChatCompletionRequest(
**base,
tool_choice={"type": "function", "function": {"name": "lookup_weather"}},
)
```
Both constructions raise a Pydantic validation error saying the input should be `auto`, `none`, or `any`. The same payloads sent to `/v1/chat/completions` return HTTP 422.
### Expected behavior
`OpenAISpec` should accept the standard `"required"` literal and named-function object, preserve them on `ChatCompletionRequest`, and pass them through the existing request/context path. Existing `"auto"`, `"none"`, and `"any"` behavior should remain compatible.
### Proposed scope
Extend only the `tool_choice` request schema and add focused CPU-only request-model and endpoint regressions. This does not change tool execution or selection inside user code.
### Environment
- LitServe `main` at `7675990`
- Python 3.12
- CPU only; no model download, GPU, or hosted service required
I used Codex assistance to investigate and prepare this report. No maintainer agreement or human review is claimed.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with ChatCompletionRequest in litserve.specs.openai and trace the /v1/chat/completions request path through LitAPI.decode_request. Add focused CPU-only request-model and endpoint regressions for required and named function tool_choice values, while confirming auto, none, and any remain compatible.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 82/100