Split compliance prompt into system instructions + typed evaluation context
- Linguagem predominante
- HTML
- Estrelas
- 5
- Forks
- 2
- Merge médio
- 1d 7h
- PRs com merge (30d)
- 1
Descrição
# Context
The compliance evaluation prompt at `backend/app/prompt_templates/compliance_verification.md` is currently rendered as a single Jinja template that interleaves static instructions (Role, Verification Protocol, Constraints) with dynamic per-requirement data. The dynamic data is assembled by five formatter helpers in `backend/app/services/compliance.py` that each emit a bullet-list string for the template:
- `get_requirement_dictionary`
- `get_general_exemptions`
- `get_exemptions`
- `get_applicability_conditions`
- `get_requirement_provisions` (which also tacks on `guidance_en` as a trailing line)
Two consequences:
- The static instructions are re-rendered into the user message on every evaluation, so prompt caching never sees a stable prefix.
- Per-requirement `guidance_en` is appended at the bottom of the Requirement section and is regularly ignored by the model — e.g. requirements whose guidance says "flag as INCONCLUSIVE if no registration number" are returning definitive non-compliance verdicts instead.
# Outcome
- Static prompt content (Role, Verification Protocol, Constraints) is sent as a system message and is byte-stable across every requirement evaluation.
- Per-call legal context and label data are delivered to the model as a single typed object, replacing the per-section string formatters.
- Per-requirement guidance is a first-class field that the verification protocol explicitly references and applies.
- Jinja-based prompt rendering is removed; the prompt becomes a plain markdown file loaded as text.
# Acceptance criteria
- The Role / Verification Protocol / Constraints content is delivered as a system message that is identical across every call to `evaluate_requirement`.
- The dynamic inputs — dictionary, general exemptions, exemptions, applicability conditions, requirement provisions, per-requirement guidance, and label data — are delivered as a single structured payload (Pydantic model serialized to JSON).
- The verification protocol's vocabulary matches the field names in the structured payload — no mismatch between section names referenced in the protocol (e.g. "Dictionary") and the JSON keys delivered to the model.
- The five formatter helpers listed above are removed from `services/compliance.py`.
- Jinja-based prompt rendering is removed: `render_prompt()` in `services/compliance.py`, the `prompt_template_env` computed property in `config.py`, and any settings exclusively supporting prompt templating are deleted. Email template rendering is unaffected.
- Per-requirement guidance reaches the model in a way the verification protocol acknowledges, such that a requirement whose guidance directs an INCONCLUSIVE verdict under specific conditions produces INCONCLUSIVE when those conditions are met.
- List ordering within the structured payload is deterministic across runs.
- Existing tests covering `evaluate_requirement` continue to pass, with updates only where the payload shape demands them.
Guia de contribuição
Nenhum guia de contribuição indexado para este repositório
Avaliação
Esta issue ainda não foi avaliada.