andreagrandi / andreagrandi/draftomen

Re-validate stored enrichment work when a validator changes

未关闭
#486 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
draftomen enhancement size: M
主要语言
Python
星标
0
派生
0
平均合并
1 小时 23 分钟
30 天内合并 PR
181

描述

## Problem

The durable set-enrichment work store derives `WorkState.COMPLETED` from the presence of a result artifact, and `WorkIdentity` keys on the contract version, prompt, schema and model hashes only. A change to a *validator* therefore leaves previously stored results in place: a resumed run reuses the old parse and reports it as current, so a validator fix silently appears to have no effect on any work that was already recorded.

Measured during the #472 paid verification: after relaxing the guide-claim card-reference binding, the harness re-run reused all 613 stored records and still reported the old verdict (0 of 7 required mechanics). Only removing the stored guide *result* artifact — keeping the durable response — made the service take its documented `UNVALIDATED` path, re-parse the already-paid response locally and change the verdict to 4 of 7 at zero additional cost.

## Proposed change

Make validator identity part of durability: include the version of the validator that produced a result in the work identity (or record it in the result artifact) so that a validator change re-derives the affected results from their durable responses instead of reusing stale parses. Re-derivation must never issue a replacement paid request, and unchanged validators must keep reusing stored results without re-parsing.

## Acceptance Criteria

- [ ] AC1: `tests/test_set_enrichment_work.py` verifies that stored work recorded under one validator version is not reported as current after the validator version changes, and is re-derivable from its stored response without a new request.
- [ ] AC2: `tests/test_set_enrichment.py` verifies a resumed run under a changed validator re-parses the durable response, records the new result and issues no additional completion call, while an unchanged validator still reuses the stored result without re-parsing.
- [ ] AC3: `tests/test_set_enrichment.py` verifies a validator change that flips an acceptance-relevant outcome (for example a formerly rejected claim becoming a retained uncertain claim) is reflected in the resumed run's result.

## Dependencies

None. Related: #472 (the verification that exposed this).

## Implementation classification

- **Estimated size:** M
- **Orchestration risk:** High
- **Reason:** Changes the durability contract and resumption semantics that every paid enrichment run depends on, so it needs explicit reuse, re-derivation and no-replacement-request coverage.

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。