antirez / antirez/ds4

Add support for structured outputs

未关闭
#210 12 条评论 3 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
C
星标
22.3k
派生
2.1k
平均合并
1 天 3 小时
30 天内合并 PR
4

描述

As in https://docs.vllm.ai/en/latest/features/structured_outputs/#online-serving-openai-api and https://developers.openai.com/api/docs/guides/structured-outputs

I tried it with `ds4` using the latest code from `main` but it doesn't seem to be working fine (both DGX Spark and m3 max) -- here's the test:

```python
#!/usr/bin/env python3
from openai import OpenAI
from pydantic import BaseModel, ConfigDict

BASE_URL = "http://127.0.0.1:8000/v1"
API_KEY = "not-needed"
MODEL = "default"

class CalendarEvent(BaseModel):
model_config = ConfigDict(extra="forbid")

name: str
date: str
participants: list[str]

def main() -> None:
client = OpenAI(base_url=BASE_URL, api_key=API_KEY)

print(f"[so-test] base_url={BASE_URL}")
print(f"[so-test] model={MODEL}")
print("[so-test] calling client.responses.parse(..., text_format=CalendarEvent)")

response = client.responses.parse(
model=MODEL,
input=[
{"role": "system", "content": "Extract the event information."},
{
"role": "user",
"content": "Alice and Bob are going to a science fair on Friday.",
},
],
text_format=CalendarEvent,
)

event = response.output_parsed
print("[so-test] parsed object:")
print(event.model_dump_json(indent=2))
print("[so-test] raw response id:", getattr(response, "id", None))

if __name__ == "__main__":
main()
```

Output:

```bash
$ python3 so-test-cli.py
[so-test] base_url=http://127.0.0.1:8000/v1
[so-test] model=default
[so-test] calling client.responses.parse(..., text_format=CalendarEvent)
Traceback (most recent call last):
[..]
pydantic_core._pydantic_core.ValidationError: 6 validation errors for CalendarEvent
people
Extra inputs are not permitted [type=extra_forbidden, input_value=['Alice', 'Bob'], input_type=list]
For further information visit https://errors.pydantic.dev/2.13/v/extra_forbidden
event
Extra inputs are not permitted [type=extra_forbidden, input_value='science fair', input_type=str]
For further information visit https://errors.pydantic.dev/2.13/v/extra_forbidden
day
Extra inputs are not permitted [type=extra_forbidden, input_value='Friday', input_type=str]
For further information visit https://errors.pydantic.dev/2.13/v/extra_forbidden
name
Field required [type=missing, input_value={'people': ['Alice', 'Bob... fair', 'day': 'Friday'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
date
Field required [type=missing, input_value={'people': ['Alice', 'Bob... fair', 'day': 'Friday'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
participants
Field required [type=missing, input_value={'people': ['Alice', 'Bob... fair', 'day': 'Friday'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
```

A bit swamped but happy to look into this at some point

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。