NetLogo / NetLogo/Netlogo-LLM-Extension
bug: llm:choose fails on models that reject json_schema instead of falling back
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 1
- Forks
- 0
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 4
Description
Problem
Since #74, llm:choose always sends its options as a strict json_schema response format on OpenAI-compatible providers (OpenAICompatibleProvider.applyResponseFormat). Models that do not support structured outputs, such as gpt-3.5-turbo, or providers that validate the format server-side, reject the request with HTTP 400. The bare-text fallback in extractEnumChoice only helps when a provider ignores the format; it cannot recover from an HTTP rejection, so a choice that worked before #74 now errors.
Found by a Codex review on 2026-09-08. The Groq-specific prompt/schema mismatch was fixed in #77; this is the remaining unsupported-format case.
Fix
Either gate native constraints on a per-model capability in models.yaml, or catch a format-rejection 400 (unsupported/response_format in the error body) and retry the same request unconstrained through the prompted path. The retry is smaller and self-healing.
Test
A stub backend that returns 400 for any request carrying response_format, then 200 for the retry. Current degradation tests only simulate a provider ignoring the format.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with OpenAICompatibleProvider.applyResponseFormat and the existing extractEnumChoice degradation tests; inspect how a provider error is surfaced and how the prompted path is reached. Add the stub backend behavior described in the issue, then verify that a 400 response for response_format is retried unconstrained and succeeds on the 200 response.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- api, backend, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100