AB-Law / AB-Law/Vett

Add evidence-grounded citations to score output

未关闭
#4 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
enhancement
主要语言
Python
星标
0
派生
0
PR 合并指标
30 天内没有已合并 PR

描述

Problem
The `score` endpoint currently returns a compact payload (`fit_score`, `matched_keywords`, `missing_keywords`, `gap_analysis`, `rewrite_suggestions`) with no verifiable trace of why each recommendation was made. The current flow in `backend/app/services/llm.py` and `backend/app/routers/score.py` does not request or persist evidence spans tied to CV sections or JD phrases, so users cannot inspect the rationale behind each score item. This makes recommendations hard to trust and difficult to audit.

Impact
Users cannot validate why a recommendation was generated when a CV section is weakly matched or missing. This weakens explainability in the `ScoreJD` UI and undermines confidence in `fit_score` for decision-making. It also blocks future quality/debug workflows where model behavior changes need to be explained to QA, hiring, or users.

Proposed Fix
For each score row/item in the score result payload, add evidence attachments from the local LLM pass:

- Extend the LLM `SCORE_PROMPT` and parsing logic to return evidence records alongside existing fields.
- Require at least one `cv_citation` (section id + line range + short snippet) and one `jd_phrase_citation` for each scored item/recommendation it was derived from.
- Use short snippets (<=1-2 lines) in payload fields designed for UI display.
- Persist and return these citations in `ScoreResponse` as additive fields (or via a versioned `score_payload_version` if backward compatibility requires full split).
- Include explicit `reason` / `evidence_missing_reason` metadata when evidence cannot be produced, rather than silently omitting entries.
- Add/update backend tests for prompt contract parsing and for API response shape (including citation presence expectations).
- Add/update frontend `ScoreJD.tsx` explainability view to render the citation block beside score outputs and keep existing cards functional when evidence is partial/missing.

Acceptance criteria to cover:
1) Each scored recommendation includes at least one CV citation and one JD phrase citation when derived from those sources.
2) Snippets are concise and include stable identifiers (section IDs and line ranges).
3) UI displays evidence in explainability alongside score metrics.
4) Missing evidence is explicit via a reason/debug field.
5) Tests assert evidence attachment behavior in score output.

Functionality Impact
The response schema change should be backward-compatible by adding optional fields to score payloads and consuming clients tolerant of older fields, or by versioning the score format under a new schema key. `agent_plan`, `matched_keywords`, `missing_keywords`, and existing score card UI behavior should continue to work as-is, with new evidence details shown in the explainability view where available.

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。