Lightning-AI / Lightning-AI/LitServe

OpenAISpec rejects standard required and named function tool_choice values

Open Beginner friendly
#745 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3.9k
Forks
304
Avg merge
3d 13h
Merged PRs (30d)
6

Description

## 🐛 Bug

`OpenAISpec` rejects two standard Chat Completions `tool_choice` values during request validation:

- `"required"`
- a named function choice such as `{"type": "function", "function": {"name": "lookup_weather"}}`

The current request model accepts only `"auto"`, `"none"`, and the non-standard compatibility value `"any"`. As a result, clients using either standard form receive HTTP 422 before `LitAPI.decode_request` runs. OpenAI's current [Chat Completions API reference](https://developers.openai.com/api/reference/cli/resources/chat) documents both `"required"` and a named function object in `tool_choice`.

### Reproduction

On current `main` (`7675990`):

```python
from litserve.specs.openai import ChatCompletionRequest

base = {
"model": "lit",
"messages": [{"role": "user", "content": "Use the weather tool"}],
}

ChatCompletionRequest(**base, tool_choice="required")
ChatCompletionRequest(
**base,
tool_choice={"type": "function", "function": {"name": "lookup_weather"}},
)
```

Both constructions raise a Pydantic validation error saying the input should be `auto`, `none`, or `any`. The same payloads sent to `/v1/chat/completions` return HTTP 422.

### Expected behavior

`OpenAISpec` should accept the standard `"required"` literal and named-function object, preserve them on `ChatCompletionRequest`, and pass them through the existing request/context path. Existing `"auto"`, `"none"`, and `"any"` behavior should remain compatible.

### Proposed scope

Extend only the `tool_choice` request schema and add focused CPU-only request-model and endpoint regressions. This does not change tool execution or selection inside user code.

### Environment

- LitServe `main` at `7675990`
- Python 3.12
- CPU only; no model download, GPU, or hosted service required

I used Codex assistance to investigate and prepare this report. No maintainer agreement or human review is claimed.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with ChatCompletionRequest in litserve.specs.openai and trace the /v1/chat/completions request path through LitAPI.decode_request. Add focused CPU-only request-model and endpoint regressions for required and named function tool_choice values, while confirming auto, none, and any remain compatible.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
82/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.