posit-dev / posit-dev/commons

Reference provenance from earlier trusted results in follow-up answers

Open
#189 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
44
Forks
1
Avg merge
1d 7h
Merged PRs (30d)
142

Description

(not necessarily intending that this be tackled right now)

Not attaching a provenance marker when no data tool was used in the current exchange makes sense, since tool usage is the basis for assigning provenance.

However, the absence of a marker kind of makes answers appear less trustworthy than even the "Untrusted" tagged answers. I think people might read it as, “we have no idea where this answer came from.”

This feels especially misleading when the agent is directly referring to the output of an earlier trusted calculation. In this screenshot, the follow-up immediately follows the measure result, so the relationship is pretty clear. If the follow-up happens much later in the conversation, though, it would be useful for the answer to reference that earlier measure result and its provenance.

I'm not thinking that follow-ups would get a trusted tag, but that they would somehow reference the turn that included the data tool with that tag.

Image

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing how provenance markers are currently assigned to data-tool results and how later follow-up answers are represented. Clarify how a follow-up should reference an earlier trusted result without receiving its own trusted tag, including behavior when the reference is much later in the conversation. Done means the behavior and scope are specified well enough to implement and test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.