antirez / antirez/ds4

Add support for structured outputs

Đang mở
#210 12 bình luận 3 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
C
Star
22.3k
Fork
2.1k
Merge trung bình
1 ngày 3 giờ
Pull request đã merge (30 ngày)
4

Mô tả

As in https://docs.vllm.ai/en/latest/features/structured_outputs/#online-serving-openai-api and https://developers.openai.com/api/docs/guides/structured-outputs

I tried it with `ds4` using the latest code from `main` but it doesn't seem to be working fine (both DGX Spark and m3 max) -- here's the test:

```python
#!/usr/bin/env python3
from openai import OpenAI
from pydantic import BaseModel, ConfigDict

BASE_URL = "http://127.0.0.1:8000/v1"
API_KEY = "not-needed"
MODEL = "default"

class CalendarEvent(BaseModel):
model_config = ConfigDict(extra="forbid")

name: str
date: str
participants: list[str]

def main() -> None:
client = OpenAI(base_url=BASE_URL, api_key=API_KEY)

print(f"[so-test] base_url={BASE_URL}")
print(f"[so-test] model={MODEL}")
print("[so-test] calling client.responses.parse(..., text_format=CalendarEvent)")

response = client.responses.parse(
model=MODEL,
input=[
{"role": "system", "content": "Extract the event information."},
{
"role": "user",
"content": "Alice and Bob are going to a science fair on Friday.",
},
],
text_format=CalendarEvent,
)

event = response.output_parsed
print("[so-test] parsed object:")
print(event.model_dump_json(indent=2))
print("[so-test] raw response id:", getattr(response, "id", None))

if __name__ == "__main__":
main()
```

Output:

```bash
$ python3 so-test-cli.py
[so-test] base_url=http://127.0.0.1:8000/v1
[so-test] model=default
[so-test] calling client.responses.parse(..., text_format=CalendarEvent)
Traceback (most recent call last):
[..]
pydantic_core._pydantic_core.ValidationError: 6 validation errors for CalendarEvent
people
Extra inputs are not permitted [type=extra_forbidden, input_value=['Alice', 'Bob'], input_type=list]
For further information visit https://errors.pydantic.dev/2.13/v/extra_forbidden
event
Extra inputs are not permitted [type=extra_forbidden, input_value='science fair', input_type=str]
For further information visit https://errors.pydantic.dev/2.13/v/extra_forbidden
day
Extra inputs are not permitted [type=extra_forbidden, input_value='Friday', input_type=str]
For further information visit https://errors.pydantic.dev/2.13/v/extra_forbidden
name
Field required [type=missing, input_value={'people': ['Alice', 'Bob... fair', 'day': 'Friday'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
date
Field required [type=missing, input_value={'people': ['Alice', 'Bob... fair', 'day': 'Friday'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
participants
Field required [type=missing, input_value={'people': ['Alice', 'Bob... fair', 'day': 'Friday'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
```

A bit swamped but happy to look into this at some point

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.