posit-dev / posit-dev/commons

Inherited ellmer methods answer without the turn rules or a provenance marker

Open
#321 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug r
Dominant language
Python
Stars
44
Forks
1
Avg merge
1d 7h
Merged PRs (30d)
142

Description

Commons inherits from ellmer::Chat (commons.R:213-215) and overrides only two of the six methods that can produce an answer. chat() and stream_async() are overridden. stream(), chat_async(), chat_structured() and chat_structured_async() are inherited without change.

This means that if a user constructs a Commons instance and uses one of those inherited methods, they don't get any of the commons-y goodness, since the turn rules never run. stream() and chat_async() skip the turn reminder, the restore reminder, and the conversation-turn span, which chat() installs at commons.R:309-318. They also skip the citation scan and the provenance aside, which only stream_async() adds. An answer from commons$stream() therefore reaches the reader with no provenance marker and no verified citations, and the trajectory record for that turn has no span.

This is probably worth consideration if we want to support those other methods, or (if not) should we prohibit them from being used in Commons?

Two parts of this are worth separating:

  • stream() and chat_async() produce prose for a person to read. These are the methods that need the turn rules.
  • chat_structured() and chat_structured_async() return parsed data. A provenance marker appended to their output would make the JSON invalid, so the correct behavior for them is probably to abort with a message that names stream_async(), not to scan.

This is an artifact of inheriting directly from ellmer on the R side. The Python package instead composes an object from chatlas instead of inheriting from it, so we're using an object that we explicitly provide methods on. The underlying chatlas client is reachable only through the documented client attribute, which cleraly documents that it skips the scanner and the marker. Composition makes the marked path the only path. Inheritance makes it a subset of the surface, and the subset needs a new override every time ellmer adds a method.

This came up while building the Python chat adapter for the UI layer, and I figured it was worth opening an issue for consideration.

@simonpcouch

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with pkg-r/R/commons.R:213-215 and 309-318, then compare the overridden chat() and stream_async() paths with inherited stream(), chat_async(), chat_structured(), and chat_structured_async(). Determine which methods Commons should support or reject; done means supported prose paths handle the documented turn, citation, provenance, and span behavior while structured paths preserve valid parsed output or clearly direct users to stream_async().

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
ai, backend-api-design
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.