antirez / antirez/ds4

Add support for structured outputs

Abierto
#210 12 comentarios 3 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
C
Estrellas
22.3k
Forks
2.1k
Merge medio
1 d 3 h
PR fusionados (30 d)
4

Descripción

As in https://docs.vllm.ai/en/latest/features/structured_outputs/#online-serving-openai-api and https://developers.openai.com/api/docs/guides/structured-outputs

I tried it with `ds4` using the latest code from `main` but it doesn't seem to be working fine (both DGX Spark and m3 max) -- here's the test:

```python
#!/usr/bin/env python3
from openai import OpenAI
from pydantic import BaseModel, ConfigDict

BASE_URL = "http://127.0.0.1:8000/v1"
API_KEY = "not-needed"
MODEL = "default"

class CalendarEvent(BaseModel):
model_config = ConfigDict(extra="forbid")

name: str
date: str
participants: list[str]

def main() -> None:
client = OpenAI(base_url=BASE_URL, api_key=API_KEY)

print(f"[so-test] base_url={BASE_URL}")
print(f"[so-test] model={MODEL}")
print("[so-test] calling client.responses.parse(..., text_format=CalendarEvent)")

response = client.responses.parse(
model=MODEL,
input=[
{"role": "system", "content": "Extract the event information."},
{
"role": "user",
"content": "Alice and Bob are going to a science fair on Friday.",
},
],
text_format=CalendarEvent,
)

event = response.output_parsed
print("[so-test] parsed object:")
print(event.model_dump_json(indent=2))
print("[so-test] raw response id:", getattr(response, "id", None))

if __name__ == "__main__":
main()
```

Output:

```bash
$ python3 so-test-cli.py
[so-test] base_url=http://127.0.0.1:8000/v1
[so-test] model=default
[so-test] calling client.responses.parse(..., text_format=CalendarEvent)
Traceback (most recent call last):
[..]
pydantic_core._pydantic_core.ValidationError: 6 validation errors for CalendarEvent
people
Extra inputs are not permitted [type=extra_forbidden, input_value=['Alice', 'Bob'], input_type=list]
For further information visit https://errors.pydantic.dev/2.13/v/extra_forbidden
event
Extra inputs are not permitted [type=extra_forbidden, input_value='science fair', input_type=str]
For further information visit https://errors.pydantic.dev/2.13/v/extra_forbidden
day
Extra inputs are not permitted [type=extra_forbidden, input_value='Friday', input_type=str]
For further information visit https://errors.pydantic.dev/2.13/v/extra_forbidden
name
Field required [type=missing, input_value={'people': ['Alice', 'Bob... fair', 'day': 'Friday'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
date
Field required [type=missing, input_value={'people': ['Alice', 'Bob... fair', 'day': 'Friday'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
participants
Field required [type=missing, input_value={'people': ['Alice', 'Bob... fair', 'day': 'Friday'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
```

A bit swamped but happy to look into this at some point

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.