NetLogo / NetLogo/Netlogo-LLM-Extension

bug: llm:choose fails on models that reject json_schema instead of falling back

Open
#81 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Scala
Stars
1
Forks
0
Avg merge
3d 1h
Merged PRs (30d)
4

Description

Problem

Since #74, llm:choose always sends its options as a strict json_schema response format on OpenAI-compatible providers (OpenAICompatibleProvider.applyResponseFormat). Models that do not support structured outputs, such as gpt-3.5-turbo, or providers that validate the format server-side, reject the request with HTTP 400. The bare-text fallback in extractEnumChoice only helps when a provider ignores the format; it cannot recover from an HTTP rejection, so a choice that worked before #74 now errors.

Found by a Codex review on 2026-09-08. The Groq-specific prompt/schema mismatch was fixed in #77; this is the remaining unsupported-format case.

Fix

Either gate native constraints on a per-model capability in models.yaml, or catch a format-rejection 400 (unsupported/response_format in the error body) and retry the same request unconstrained through the prompted path. The retry is smaller and self-healing.

Test

A stub backend that returns 400 for any request carrying response_format, then 200 for the retry. Current degradation tests only simulate a provider ignoring the format.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with OpenAICompatibleProvider.applyResponseFormat and the existing extractEnumChoice degradation tests; inspect how a provider error is surfaced and how the prompted path is reached. Add the stub backend behavior described in the issue, then verify that a 400 response for response_format is retried unconstrained and succeeds on the 200 response.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
api, backend, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.