Bug with sglang with how the tool call is passed
- Dominant language
- No language data
- Stars
- 2.1k
- Forks
- 153
- PR merge metrics
- No merged PRs in 30d
Description
using custom model provider endpoint of hscn and compactation that was initialized by the model not user initilaized is failing .
Compaction failed. CAPIError: 400 2 validation errors:
{'type': 'literal_error', 'loc': ('body', 'tool_choice', 'ToolChoice', 'type'), 'msg': "Input should be 'function'", 'input': 'none', 'ctx': {'expected': "'function'"}}
{'type': 'literal_error', 'loc': ('body', 'tool_choice', "literal['auto','required','none']"), 'msg': "Input should be 'auto', 'required' or 'none'", 'input': {'function': {'name': ''}, 'type': 'none'}, 'ctx': {'expected': "'auto', 'required' or 'none'"}}
File "/usr/local/lib/python3.12/dist-packages/sglang/srt/entrypoints/http_server.py", line 1491, in openai_v1_chat_completions
POST /v1/chat/completions [{'type': 'literal_error', 'loc': ('body', 'tool_choice', 'ToolChoice', 'type'), 'msg': "Input should be 'function'", 'input': 'none', 'ctx': {'expected': "'function'"}}, {'type': 'literal_error', 'loc': ('body', 'tool_choice', "literal['auto','required','none']"), 'msg': "Input should be 'auto', 'required' or 'none'", 'input': {'function': {'name': ''}, 'type': 'none'}, 'ctx': {'expected': "'auto', 'required' or 'none'"}}]
but when i do /compact manually it works .
idk why .
---
| Field | Value |
| --- | --- |
| App version | 1.1.1 |
| OS | Windows 10.0.26200 |
| Theme | GitHub |
| Path | /chat |
| Tenure | Week 6 |
Contributor guide
Research direction
Start with the reported /compact behavior and compare it with the failing model-initialized compaction request. Inspect sglang/srt/entrypoints/http_server.py at openai_v1_chat_completions and reproduce the 400 validation error using the custom hscn provider; done means model-initialized compaction passes a valid tool_choice while manual /compact continues to work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100