feat: structured-output repair loop on validation failure

Motivation

Pydantic AI's validation-retry loop (ModelRetry, described at https://ai.pydantic.dev/output/) is one of its most-cited primitives: when the model's output fails Pydantic validation (or a user-supplied validator raises ModelRetry), the validation error is fed back to the model as a correction message and it gets another shot, up to a configurable number of attempts. Chatlas has no equivalent. Chat.chat_structured() (chatlas/_chat.py) calls data_model.model_validate(dat) on whatever JSON came back from _submit_and_extract_data() — if that raises pydantic.ValidationError, it propagates straight to the caller with no chance for the model to self-correct. The same is true of parallel_chat_structured() in chatlas/_parallel.py, which does its own data_model.model_validate(dat) per conversation.

This is a distinct failure mode from truncation: chatlas just landed check_finish_reason() in chatlas/_turn.py (gh-315 / PR #344, merged to main as commit 9b5755a — not yet in this branch), which raises a clear ValueError before parsing when finish_reason is max_tokens/context_window/content_filter for a requested data model. That fix is about "nothing useful can be done with a partial response" and is out of scope here. This issue is about the case where the model did return complete, parseable JSON that simply doesn't satisfy the schema (wrong type, missing required field, a custom @field_validator rejecting a value) — a case a retry-with-feedback loop can often fix.

Proposed approach

import chatlas as ctl
from pydantic import BaseModel, field_validator

class Person(BaseModel):
    name: str
    age: int

    @field_validator("age")
    @classmethod
    def must_be_reasonable(cls, v):
        if not (0 < v < 130):
            raise ValueError(f"age {v} is not plausible")
        return v

chat = ctl.ChatOpenAI()
# On pydantic.ValidationError, chatlas appends the error as a user-turn
# correction (e.g. "Your last response didn't match the schema: <error>.
# Please try again.") and resubmits, up to `max_retries` times.
person = chat.chat_structured("Rin, age 900", data_model=Person, max_retries=2)

Candidate implementation: wrap the data_model.model_validate(dat) call in Chat.chat_structured()/.chat_structured_async() (and the Chat._extract_turn_json() + model_validate() pattern reused in parallel_chat_structured()/batch_chat_structured()) in a loop that catches pydantic.ValidationError, builds a corrective UserTurn from err.errors() (Pydantic's structured error list gives field paths + messages, which read well as feedback), and resubmits via the same _submit_turns/_submit_turns_async path already used internally. Consider a validator: Callable[[dict], None] | None escape hatch so users can raise on custom business-rule failures beyond what the Pydantic schema encodes, going through the same retry loop.

Provider interaction to check: for Anthropic, _provider_anthropic.py's structured_output_mode ("auto"/"native"/"tool") means native structured outputs (JSON-schema-constrained) rarely produce schema-invalid JSON, but the tool-based fallback (and any custom @field_validator) can still fail — the repair loop should apply uniformly regardless of structured_output_mode since validation happens after the provider-specific extraction, in chat_structured() itself.

Alternatives / prior art

• Pydantic AI: ModelRetry / output validators (https://ai.pydantic.dev/output/).
• ellmer: no equivalent found — chat_structured()/type_object() in ellmer raise directly on invalid output as well (checked R/ source and searched tidyverse/ellmer issues for "retry"/"validation"/"structured output" prior art; nothing on point). This would be a chatlas-specific addition, not one aligning with existing ellmer behavior — worth flagging upstream if it proves valuable.
• Related, but distinct: gh-315 / PR #344 (check_finish_reason) handles truncated/incomplete responses, not schema-validity of complete responses.

Open questions

• Default max_retries? Pydantic AI defaults to 1 retry; given cost concerns (see the sibling retry/fallback issue), a conservative default (1) with opt-out (max_retries=0) seems safest.
• Should the correction be a new UserTurn (visible in get_turns()/history) or an ephemeral resubmission that doesn't pollute the conversation record? ellmer's and Pydantic AI's approaches both keep it in-context since the model needs to see its own bad output to correct it.
• Should this also apply to extract_data() (deprecated but still present) or only the current chat_structured()?

Drafted from a competitive review of llm / Pydantic AI / LangChain / LiteLLM (July 2026); filed via Claude Code on behalf of @cpsievert.

Start with Chat.chat_structured() and chat_structured_async() in chatlas/_chat.py, then compare the extraction and validation flow in chatlas/_parallel.py and the provider handling in chatlas/_provider_anthropic.py. Review the existing _submit_turns paths and UserTurn representation before resolving the open questions around retry defaults, history visibility, validators, and extract_data(). Done should cover validation failures consistently across the structured-output entry points without changing truncation handling.