andreagrandi / andreagrandi/draftomen

Version the narration prompt and output admission policy

Open
#455 0 comments 0 reactions 0 assignees View on GitHub
draftomen enhancement size: M
Dominant language
Python
Stars
0
Forks
0
Avg merge
1h 23m
Merged PRs (30d)
181

Description

## Problem

Unvalidated generated text can invent claims or violate the concise prose contract; validating only after display is too late for the checks that protect streamed output.

## Proposed change

Implement the versioned application-owned prompt and the incremental output admission strategy selected by #454. Require supplied facts only, concise English, preserved numbers, and confidence/uncertainty grounded in engine reasons. Add bounded length, unexpected-formatting, unknown-number, empty-output, and malformed-output handling, including valid numeric card names. Expose accepted text and rejection outcomes to the runtime/session without embedding a second recommendation engine. Document the semantic limits of mechanical checks.

## Acceptance Criteria

- [ ] AC1: Corpus evaluation confirms the prompt expresses all PRD factuality/prose constraints and uses only #452 facts.
- [ ] AC2: Output-boundary tests reject unsupplied numeric claims, prohibited formatting, excessive output, and empty completions while accepting valid names and metrics.
- [ ] AC3: Chunk-boundary tests show an unsafe numeric/format sequence split across tokens is not admitted merely because individual tokens appear harmless.
- [ ] AC4: A streamed corpus experiment produces progressive admitted text under the #454 policy and records rejection/fallback behavior without claiming semantic certainty from mechanical validation.
- [ ] AC5: Prompt/policy version changes are identifiable in evaluation results and can be compared against the fixed corpus; verified by a versioned evaluation run.

## Implementation classification

- **Estimated size:** M
- **Orchestration risk:** High
- **Reason:** Streaming and factual admission interact at boundaries ordinary happy-path tests miss.

## Dependencies

Direct prerequisites: #454. Consumer: Run bounded local narration in a reusable worker.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.