graniet / graniet/llm

Issue with Google structured response

Open
#44 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
365
Forks
90
PR merge metrics
No merged PRs in 30d

Description

To Reproduce:
A request with the schema property fails with a 400
One without succeeds.
Same API Key same code just removed the schema property on the request
```rs
let builder = LLMBuilder::new()
.backend(self.model.provider())
.api_key(&self.api_key)
.model(self.model.to_string())
.max_tokens(self.max_tokens)
.temperature(self.temperature)
.stream(false)
.system("DO thing")
.schema(output_schema);
```
fails with 400

```rs
let builder = LLMBuilder::new()
.backend(self.model.provider())
.api_key(&self.api_key)
.model(self.model.to_string())
.max_tokens(self.max_tokens)
.temperature(self.temperature)
.stream(false)
.system("DO thing");

```

That does the thing.
This is using `gemini-2.0-flash` with which I have verified works with curl.
```sh
curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent?key=$GOOGLE_KEY" -H 'Content-Type: application/json' -d '{
"contents": [{
"parts":[
{ "text": "List a few popular cookie recipes, and include the amounts of ingredients." }
]
}],
"generationConfig": {
"responseMimeType": "application/json",
"responseSchema": {
"type": "ARRAY",
"items": {
"type": "OBJECT",
"properties": {
"recipeName": { "type": "STRING" },
"ingredients": {
"type": "ARRAY",
"items": { "type": "STRING" }
}
},
"propertyOrdering": ["recipeName", "ingredients"]
}
}
}
}' 2> /dev/null | head
```

It works for Anthropic and Deepseek v3-coder
Not tried with Open AI

Contributor guide

Open the contributing guide

Research direction

Start at the LLMBuilder schema handling and the Gemini generateContent request path, then compare the request produced by .schema(output_schema) with the working curl payload. Confirm the response schema and MIME settings are translated correctly, and verify that a Gemini request with schema succeeds without breaking the existing Anthropic and DeepSeek cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.