andreagrandi / andreagrandi/draftomen

Re-validate stored enrichment work when a validator changes

Đang mở
#486 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
draftomen enhancement size: M
Ngôn ngữ chính
Python
Star
0
Fork
0
Merge trung bình
1 giờ 23 phút
Pull request đã merge (30 ngày)
181

Mô tả

## Problem

The durable set-enrichment work store derives `WorkState.COMPLETED` from the presence of a result artifact, and `WorkIdentity` keys on the contract version, prompt, schema and model hashes only. A change to a *validator* therefore leaves previously stored results in place: a resumed run reuses the old parse and reports it as current, so a validator fix silently appears to have no effect on any work that was already recorded.

Measured during the #472 paid verification: after relaxing the guide-claim card-reference binding, the harness re-run reused all 613 stored records and still reported the old verdict (0 of 7 required mechanics). Only removing the stored guide *result* artifact — keeping the durable response — made the service take its documented `UNVALIDATED` path, re-parse the already-paid response locally and change the verdict to 4 of 7 at zero additional cost.

## Proposed change

Make validator identity part of durability: include the version of the validator that produced a result in the work identity (or record it in the result artifact) so that a validator change re-derives the affected results from their durable responses instead of reusing stale parses. Re-derivation must never issue a replacement paid request, and unchanged validators must keep reusing stored results without re-parsing.

## Acceptance Criteria

- [ ] AC1: `tests/test_set_enrichment_work.py` verifies that stored work recorded under one validator version is not reported as current after the validator version changes, and is re-derivable from its stored response without a new request.
- [ ] AC2: `tests/test_set_enrichment.py` verifies a resumed run under a changed validator re-parses the durable response, records the new result and issues no additional completion call, while an unchanged validator still reuses the stored result without re-parsing.
- [ ] AC3: `tests/test_set_enrichment.py` verifies a validator change that flips an acceptance-relevant outcome (for example a formerly rejected claim becoming a retained uncertain claim) is reflected in the resumed run's result.

## Dependencies

None. Related: #472 (the verification that exposed this).

## Implementation classification

- **Estimated size:** M
- **Orchestration risk:** High
- **Reason:** Changes the durability contract and resumption semantics that every paid enrichment run depends on, so it needs explicit reuse, re-derivation and no-replacement-request coverage.

Hướng dẫn đóng góp

Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.