`response` field has incorrect type when using JSON Mode

# Current Issue

See https://github.com/cloudflare/cf-platform-issues/issues/18#issuecomment-5319666039

# Original Issue

### What versions & operating system are you using?

```

System:
OS: macOS 15.7.1
CPU: (12) arm64 Apple M2 Pro
Memory: 149.84 MB / 16.00 GB
Shell: 5.9 - /bin/zsh
Binaries:
Node: 24.11.0 - /Users/gamer/.nvm/versions/node/v24.11.0/bin/node
npm: 11.6.1 - /Users/gamer/.nvm/versions/node/v24.11.0/bin/npm
pnpm: 10.28.2 - /opt/homebrew/bin/pnpm
bun: 1.3.7 - /Users/gamer/.bun/bin/bun
Deno: 2.1.4 - /Users/gamer/.deno/bin/deno
npmPackages:
@cloudflare/workers-types: ^4.20260203.0 => 4.20260203.0
wrangler: ^4.62.0 => 4.62.0
```

### Please provide a link to a minimal reproduction

https://github.com/earth-app/cloud

### Describe the Bug

There seems to be issues with the `json_schema` response type and trying to use the huggingface model `@hf/nousresearch/hermes-2-pro-mistral-7b`.

Snippet from [`earth-app/cloud`](https://github.com/earth-app/cloud):

```ts
const quizModel = '@hf/nousresearch/hermes-2-pro-mistral-7b';

export type ArticleQuizQuestion = {
question: string;
} & (
| {
type: 'multiple_choice';
options: string[];
correct_answer: string;
correct_answer_index: number;
}
| {
type: 'true_false';
options: ('true' | 'false')[];
correct_answer: 'true' | 'false';
correct_answer_index: number;
is_true: boolean;
is_false: boolean;
}
);

const articleQuizAiSchema = {
type: 'object',
properties: {
questions: {
type: 'array',
minItems: 2,
maxItems: 5,
items: {
type: 'object',
properties: {
question: {
type: 'string',
maxLength: 100
},
type: { type: 'string', enum: ['multiple_choice', 'true_false'] },
options: {
type: 'array',
maxItems: 4,
items: {
type: 'string',
maxLength: 60
}
},
correct_answer: { type: 'string' },
correct_answer_index: { type: 'number' },
is_true: { type: 'boolean' },
is_false: { type: 'boolean' }
},
required: ['question', 'type', 'options', 'correct_answer', 'correct_answer_index']
}
}
},
required: ['questions']
};

const QUIZ_CUTOFF = 300;

export async function createArticleQuiz(
article: Pick,
ai: Ai
): Promise {
try {
const content = article.ocean.content || article.ocean.abstract || '';
const firstPart = content.substring(0, QUIZ_CUTOFF);
const lastPart = content.substring(content.length - QUIZ_CUTOFF);
const quizResult = await ai.run(quizModel, {
messages: [
{ role: 'system', content: prompts.articleQuizSystemMessage.trim() },
{
role: 'user',
content:
content.length > QUIZ_CUTOFF * 2
? firstPart + '... (truncated) ...' + lastPart
: content
},
{ role: 'user', content: prompts.articleQuizPrompt.trim() }
],
max_tokens: 512,
temperature: 0.3,
response_format: {
type: 'json_schema',
json_schema: articleQuizAiSchema
}
});

const parsedResult = JSON.parse(quizResult?.response || '{"questions":[]}');
const quizData = (parsedResult.questions || []) as ArticleQuizQuestion[];
return quizData;
} catch (error) {
console.error('Quiz generation failed, continuing without quiz:', error);
return []; // Return empty quiz rather than failing article creation
}
}
```

Changing `max_tokens` does not matter, nor does trying to downsize the schema with `maxLength` / `maxItems` or article content provided with `QUIZ_CUTOFF`. I still get the same `1024` error, and large cutoffs only produce smaller movements toward the 1024 cutoff.

It seems that `max_tokens` should be passed to `max_new_tokens`, but for some reason it is being ignored. This is especially true since the [docs](https://developers.cloudflare.com/workers-ai/models/hermes-2-pro-mistral-7b/) specify that the default is 256, but it always 1024.

### Please provide any relevant error logs

```
Quiz generation failed, continuing without quiz: InferenceUpstreamError [AiError]: 3025: error with TGI API: failed to parse response (invalid type: map, expected a sequence at line 1 column 0): {"error":"Input validation error: `max_new_tokens` must be <= 1024. Given: 1321","error_type":"validation"}

at Ai._parseError (cloudflare-internal:ai-api:208:24)
at async Ai.run (cloudflare-internal:ai-api:186:19)
at async createArticleQuiz
(file:///Users/gamer/gmitch215/earth-app/cloud/src/content/boat.ts:523:22)
at null. (async
file:///Users/gamer/gmitch215/earth-app/cloud/.wrangler/tmp/dev-Ftlejr/index.js:105986:22)
at [object Object]
at async Object.scheduled
(file:///Users/gamer/gmitch215/earth-app/cloud/src/scheduled.ts:36:3)
at async scheduled
(file:///Users/gamer/gmitch215/earth-app/cloud/node_modules/wrangler/templates/middleware/middleware-scheduled.ts:8:3)
at async drainBody
(file:///Users/gamer/gmitch215/earth-app/cloud/node_modules/wrangler/templates/middleware/middleware-ensure-req-body-drained.ts:5:10)
```

The reproduction calls Ai.run from src/content/boat.ts:523, with scheduling reaching it from src/scheduled.ts:36; start by tracing how response_format and max_tokens are translated for the Hermes model. Reproduce the 1321 max_new_tokens validation error, then verify that the request honors the caller's token limit and returns the expected JSON response.

索引モデルが issue の本文から書いたものです。