Tracking: competitive review workstreams (July 2026)

This is a tracking issue for a set of workstreams that came out of a July 2026 review comparing chatlas with four popular Python LLM frameworks: llm, Pydantic AI, LangChain, and LiteLLM. The goal was to identify what would make chatlas comparable — or preferable — to these options for its target user: the solo engineer / data scientist who values simplicity over over-engineering.

Headline findings

• The "simple, stateful chat" niche is getting less crowded, not more. Pydantic AI (v2 "capabilities" redesign), LangChain (agent-platform repositioning on LangGraph), and LiteLLM (Rust gateway rewrite) are all moving upmarket toward production agent platforms, accumulating exactly the complexity chatlas avoids. llm is the closest neighbor in spirit, but is CLI-first.
• The gaps worth closing are not platform features. They're a handful of small reliability and guardrail primitives that showed up independently across multiple frameworks — each buildable on machinery chatlas already has (typed turns, tool callbacks, token/cost accounting).
• Underexploited advantages worth communicating better: stateful Chat ergonomics, batch_chat()'s provider-discount support (none of the four has an equivalent), the typed/normalized content model (citations, thinking, tool events), notebook/console UX, and OpenTelemetry out of the box.

Workstreams

The sub-issues attached to this issue, roughly by priority:

High value, small surface (precedented in 3+ of the reviewed frameworks):

• #353 — retry with backoff + opt-in model fallback
• #357 — structured-output repair loop on validation failure
• #8 — limits on conversation size (now the parent of #359 usage guards and #356 history compaction, which propose concrete designs for slices of it)
• #348 — deferred (async) tool approval

Real user pain, bigger design questions:

• #352 — conversation persistence (save()/load() + optional local log)
• #358 — opt-in local response cache (discussion)

Scope decisions & DX:

• #349 — prompt "profiles" (serializable prompt + schema + params bundles)
• #350 — minimal CLI? (decision)
• #351 — thin embed() helper, or explicit non-goal? (decision)
• #354 — docs: "Chat as a tool" delegation recipe
• #355 — docs: finish and link why-chatlas.qmd

The issues marked (decision) may legitimately close as "no" — that still resolves them.

Explicit non-goals

The review reinforced these as deliberate scope boundaries, not oversights:

• No middleware/hook-pipeline system or agent graph layer. This is the most-criticized surface of LangChain, and the v1→v2 churn in Pydantic AI points the same direction. The existing on_tool_request/on_tool_result callbacks and ToolRejectError cover the practical cases at a fraction of the conceptual cost.
• No RAG primitives, vector stores, or document loaders. LangChain is itself demoting these out of its core package.
• No plugin architecture. The Provider ABC already provides extensibility without the packaging/discovery overhead (and plugin-quality variance is llm's weak spot).
• No provider-breadth chase, and no proxy/gateway/multi-tenant features. Curated, tested providers remain the strategy; an OpenAI-compatible gateway (e.g. LiteLLM) via ChatOpenAICompletions() is the documented escape hatch for the long tail.

Start with the linked sub-issues, especially the reliability primitives and scope decisions, then compare them with the headline findings and explicit non-goals here. This tracking issue is done when each workstream has been implemented, documented, or resolved as a deliberate no-go decision.