togethercomputer / togethercomputer/together-typescript
Structured Outputs not supporting meta-llama/Llama-3.2-3B-Instruct-Turbo
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 71
- Forks
- 10
- Avg merge
- 3d 10h
- Merged PRs (30d)
- 5
Description
Even thought it's stated that meta-llama/Llama-3.2-3B-Instruct-Turbo is supported for Structured Outputs, this code returns the following error:
const extract = await together.chat.completions.create({
messages: [
{
role: "system",
content:
"The following is a list of items from a menu. Only answer in JSON.",
},
{
role: "user",
content: menuItems!,
},
],
model: "meta-llama/Llama-3.2-3B-Instruct-Turbo",
// @ts-expect-error - this is not typed in the API
response_format: { type: "json_object", schema: jsonSchema },
});
Error:
error: {
id: 'npsvkwr-57nCBj-92f800485c83c4bf',
error: {
message: "Input validation error: grammar is not valid: grammar is not valid: Grammar must have a 'properties' field",
type: 'invalid_request_error',
param: null,
code: null
}
}
}
The same code works with meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with together.chat.completions.create using meta-llama/Llama-3.2-3B-Instruct-Turbo and the shown response_format, then compare it with Meta-Llama-3.1-8B-Instruct-Turbo. Trace the Structured Outputs request path in the TypeScript client and verify whether the model is actually accepted; done means the documented model accepts the same schema without the grammar validation error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100