cgreene / cgreene/farewright

Run cold-user rehearsals across legitimate inputs and outcomes

Open
#32 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

demo-integrity testing workstream:agent workstream:deployment
Dominant language
No language data
Stars
0
Forks
0
Avg merge
8h 21m
Merged PRs (30d)
2

Description

Outcome

Cold users can supply their own supported trip and mandate and receive correct factual graph/policy outcomes without facilitator steering or hidden golden values.

Scope

  • Vary routes, dates, hard constraints, soft preferences, USD caps, thresholds, and allowed actions.
  • Include autonomous, approval, prohibited, no-match, temporary-unavailability, stale-selection/refusal, partial-success, and non-USD rejection cases.
  • Verify the agent ranks/explains while Farewright only evaluates typed rules and enforces at execution.
  • Exercise the full disruption/service/settlement loop only when the user's selected action passes execution guards.
  • Ask hostile stakeholder questions about simulation, fallbacks, monitoring, PII, authority, and refunds.

Acceptance

  • Facilitator explains only supported/test boundaries and does not steer request, budget, threshold, complaint, or outcome.
  • The private reliability matrix is used for QA measurement only, not user selection.
  • At least one honest no-match and one policy-prohibited/refused run end without being reframed as failure.
  • A complete search may stop honestly; any optional recovery/service/settlement drill that proceeds uses the actual selected facts.
  • A model attempt to reuse stale autonomous evidence, supply its own budget math, or book one leg of an over-cap plan is rejected.
  • Measurements and failures are recorded without turning them into a deterministic script.

Dependencies

  • #31 live-test/outage rehearsal and #34 private reliability artifact.
  • All earlier milestone gates satisfied or explicitly documented.

Review

Inputs to the mandatory independent demo-integrity review.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the prerequisites in #31 and #34 and confirming that all earlier milestone gates are satisfied or documented. Use the issue's acceptance checklist to run varied cold-user rehearsals, record measurements and failures without a deterministic script, and provide the resulting material for the independent demo-integrity review.

Written by the indexing model from the issue text.

Assessment

Domain
testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.