Invalid Schema Error when Using examples Field in Pydantic Model for OpenAI Python Client
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 31.6k
- Forks
- 5.7k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 96
Description
Confirm this is an issue with the Python library and not an underlying OpenAI API
- This is an issue with the Python library
Describe the bug
When you add the examples field in the Field definition of a Pydantic model (as shown in the code below) while using the OpenAI Python client, it raises a BadRequestError with error code 400. The error message indicates that the examples field is not permitted in the schema for the response_format parameter.
This is the error message:
raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "Invalid schema for response_format 'MathReasoning': In context=('properties', 'final_answer'), 'examples' is not permitted.", 'type': 'invalid_request_error', 'param': 'response_format', 'code': None}}
Question:
Is this behavior due to a limitation in the OpenAI Python client or API, or is it something that could potentially be patched in the future?
To Reproduce
from openai import Client
from pydantic import BaseModel, Field
from dotenv import load_dotenv
load_dotenv()
client = Client()
class Step(BaseModel):
explanation: str
output: str
class MathReasoning(BaseModel):
steps: list[Step]
final_answer: str = Field(
title="Final Answer",
description="The final answer to the math problem",
examples=["x = -3", "x = 2"]
)
completion = client.beta.chat.completions.parse(
model="gpt-4o-2024-08-06",
messages=[
{"role": "system", "content": "You are a helpful math tutor. Guide the user through the solution step by step."},
{"role": "user", "content": "how can I solve 8x + 7 = -23"}
],
response_format=MathReasoning,
)
math_reasoning = completion.choices[0].message
# If the model refuses to respond, you will get a refusal message
if (math_reasoning.refusal):
print(math_reasoning.refusal)
else:
print(math_reasoning.parsed)
Code snippets
OS
Windows
Python version
Python v3.11.9
Library version
openai v1.68.2
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at client.beta.chat.completions.parse and inspect how the Pydantic response_format schema is sent. Reproduce the issue with the supplied MathReasoning model and determine whether the client or API rejects the examples field. Done means a documented decision on whether a client change is warranted.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100