antirez / antirez/ds4

Add support for structured outputs

オープン
#210 コメント 12 件 リアクション 3 件 担当者 0 名 GitHub で見る
主要言語
C
スター
22.3k
フォーク
2.1k
平均マージ
1日 3時間
マージ済み PR(30日)
4

説明

As in https://docs.vllm.ai/en/latest/features/structured_outputs/#online-serving-openai-api and https://developers.openai.com/api/docs/guides/structured-outputs

I tried it with `ds4` using the latest code from `main` but it doesn't seem to be working fine (both DGX Spark and m3 max) -- here's the test:

```python
#!/usr/bin/env python3
from openai import OpenAI
from pydantic import BaseModel, ConfigDict

BASE_URL = "http://127.0.0.1:8000/v1"
API_KEY = "not-needed"
MODEL = "default"

class CalendarEvent(BaseModel):
model_config = ConfigDict(extra="forbid")

name: str
date: str
participants: list[str]

def main() -> None:
client = OpenAI(base_url=BASE_URL, api_key=API_KEY)

print(f"[so-test] base_url={BASE_URL}")
print(f"[so-test] model={MODEL}")
print("[so-test] calling client.responses.parse(..., text_format=CalendarEvent)")

response = client.responses.parse(
model=MODEL,
input=[
{"role": "system", "content": "Extract the event information."},
{
"role": "user",
"content": "Alice and Bob are going to a science fair on Friday.",
},
],
text_format=CalendarEvent,
)

event = response.output_parsed
print("[so-test] parsed object:")
print(event.model_dump_json(indent=2))
print("[so-test] raw response id:", getattr(response, "id", None))

if __name__ == "__main__":
main()
```

Output:

```bash
$ python3 so-test-cli.py
[so-test] base_url=http://127.0.0.1:8000/v1
[so-test] model=default
[so-test] calling client.responses.parse(..., text_format=CalendarEvent)
Traceback (most recent call last):
[..]
pydantic_core._pydantic_core.ValidationError: 6 validation errors for CalendarEvent
people
Extra inputs are not permitted [type=extra_forbidden, input_value=['Alice', 'Bob'], input_type=list]
For further information visit https://errors.pydantic.dev/2.13/v/extra_forbidden
event
Extra inputs are not permitted [type=extra_forbidden, input_value='science fair', input_type=str]
For further information visit https://errors.pydantic.dev/2.13/v/extra_forbidden
day
Extra inputs are not permitted [type=extra_forbidden, input_value='Friday', input_type=str]
For further information visit https://errors.pydantic.dev/2.13/v/extra_forbidden
name
Field required [type=missing, input_value={'people': ['Alice', 'Bob... fair', 'day': 'Friday'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
date
Field required [type=missing, input_value={'people': ['Alice', 'Bob... fair', 'day': 'Friday'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
participants
Field required [type=missing, input_value={'people': ['Alice', 'Bob... fair', 'day': 'Friday'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
```

A bit swamped but happy to look into this at some point

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。