googleapis / googleapis/python-genai
Validating batch response with Google Search grounding against `GenerateContentResponse` causes validation error
- Dominant language
- Python
- Stars
- 4k
- Forks
- 1k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 40
Description
It seems like the `GenerateContentResponse` objects and its nested objects are not designed properly for the any reponses that uses the built-in Google Search tool.
It returns:
```
response.candidates.0.groundingMetadata.groundingChunks.0.chunkMetadata
Extra inputs are not permitted [type=extra_forbidden, input_value={'query': 'query'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.12/v/extra_forbidden
```
This is caused by the `GroundingChunk` object, which does not actually match the `groundingChunk` returned by the Vertex AI API:
```python
class GroundingMetadata(_common.BaseModel):
"""Information for various kinds of grounding."""
grounding_chunks: Optional[list[GroundingChunk]] = Field(
default=None,
description="""A list of supporting references retrieved from the grounding
source. This field is populated when the grounding source is Google
Search, Vertex AI Search, or Google Maps.
""",
)
# Truncated for brevity
class GroundingChunk(_common.BaseModel):
"""A piece of evidence that supports a claim made by the model.
This is used to show a citation for a claim made by the model. When grounding
is enabled, the model returns a `GroundingChunk` that contains a reference to
the source of the information.
"""
image: Optional[GroundingChunkImage] = Field(
default=None,
description="""A grounding chunk from an image search result. See the `Image` message for details.""",
)
maps: Optional[GroundingChunkMaps] = Field(
default=None,
description="""A `Maps` chunk is a piece of evidence that comes from Google Maps.
It contains information about a place, such as its name, address, and
reviews. This is used to provide the user with rich, location-based
information.""",
)
retrieved_context: Optional[GroundingChunkRetrievedContext] = Field(
default=None,
description="""A grounding chunk from a data source retrieved by a retrieval tool, such as Vertex AI Search. See the `RetrievedContext` message for details""",
)
web: Optional[GroundingChunkWeb] = Field(
default=None,
description="""A grounding chunk from a web page, typically from Google Search. See the `Web` message for details.""",
)
```
The actual JSON `groundingChunk` JSON:
```json
"groundingMetadata" : {
"groundingChunks" : [ {
"chunkMetadata" : {
"query" : "dummy_query"
},
"web" : {
"title" : "youtube.com",
"uri" : "https://vertexaisearch.cloud.google.com/grounding-api-redirect/id"
}
} ],
```
#### Environment details
- Programming language: Python
- OS: MacOS
- Language runtime version: 3.12
- Package version: 2.6.0
#### Steps to reproduce
1. Create batch request or requests with google search enabled and ensure the model actually performs a Google search , dummy example:
```json
{
"contents" : [ {
"parts" : [ {
"text" : "Dummy request"
} ],
"role" : "user"
} ],
"generationConfig" : {
"responseMimeType" : "application/json",
"responseSchema" : {
},
"temperature" : 0.2
},
"systemInstruction" : {
"parts" : [ {
"text" : "Dummy system instructions "
} ]
},
"tools" : [ {
"googleSearch" : {
"excludeDomains" : [ "example.com" ] // google search on batch inference. does not work unless you actually specify an exclusion domain... please fix as well
}
} ]
}
```
2. After results are ready, read the results and validate the `GenerationResponse` object againts the "response" part of the JSON response:
```python
from google.genai import types
result = types.GenerateContentResponse.model_validate_json(parsed_json_dict["response"]) # Raises validation error
```
Contributor guide
Assessment
This issue has not been assessed yet.