Azure / Azure/azure-sdk-for-python

azure-ai-inference : LangChain `AzureAIChatCompletionsModel` Does Not Support OpenAI-Style JSON Schema Format

Open
#44,201 2 comments 0 reactions 0 assignees View on GitHub
AI Model Inference customer-reported needs-team-attention question Service Attention
Dominant language
Python
Stars
5.6k
Forks
3.4k
Avg merge
1d 21h
Merged PRs (30d)
193

Description

- Issue description
When using `langchain_azure_ai.AzureAIChatCompletionsModel` with structured output (e.g., create_agent(..., response_format=ProviderStrategy()...), the integration fails with:
`ValueError: Unsupported `response_format` {'type': 'json_schema', 'json_schema': {...}}`
This occurs in `azure/ai/inference/_patch.py`

- Steps to Reproduce
```
from langchain_azure_ai.chat_models import AzureAIChatCompletionsModel
from pydantic import BaseModel, Field
import os

llm = AzureAIChatCompletionsModel(
endpoint=os.getenv("GITHUB_INFERENCE_ENDPOINT"),
credential=os.getenv("GITHUB_TOKEN"),
model="gpt-4o",
)

class ContactInfo(BaseModel):
name: str = Field(description="The name of the person")
email: str = Field(description="The email of the person")
phone: str = Field(description="The phone number of the person")

agent = create_agent(
model = gllm_4_1,
tools = [search_tool],
response_format=ProviderStrategy(ContactInfo)
)

agent.invoke({"messages": [HumanMessage(content="Extract contact info from: John Doe, john@example.com, (555) 123-4567")]})

```

- Alternatives considered
Use `langchain-openai` instead, which treats Azure-compatible endpoints as OpenAI endpoints.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.