anthropics / anthropics/claude-cookbooks
Proposal: guide on citation-faithfulness evals — catching hallucinated/unsupported citations in grounded generation
- 主要語言
- Jupyter Notebook
- 星號
- 52.7k
- 分支
- 6.3k
- 平均合併
- 25 分鐘
- 30 天內合併 PR
- 6
描述
### What I'd like to add
A single self-contained guide on **evaluating citation faithfulness** — verifying that every claim a model makes is actually supported by the source it cites, and catching *hallucinated or misattributed citations* before they reach a user. It's a failure mode nearly every RAG / grounded-generation app hits, but I couldn't find a guide covering it directly.
### Why it's useful (and not a duplicate)
I reviewed the existing evaluation material first, per the contributing guidelines:
- `capabilities/classification/evaluation` — classification accuracy
- `capabilities/knowledge_graph/evaluation` — extraction quality
- `tool_evaluation/` and `capabilities/retrieval_augmented_generation` — RAG / tool patterns
These evaluate *retrieval* and *task accuracy*, but none check whether a generated claim's **cited passage can actually be located in the source** and whether it **supports the claim**. That's the gap this guide fills.
### Rough outline
1. The problem: plausible-but-unsupported citations, and why retrieval metrics miss them.
2. An eval that, per claim: extracts the cited passage → verifies it exists verbatim in the source → uses Claude as judge to grade support level (supports / partial / contradicts / not found).
3. A small labelled dataset + scoring (precision/recall on "unsupported-citation" detection).
4. Hardening: rejecting hallucinated quotes, confidence calibration.
5. *(Optional)* a variant that pulls sources live via an MCP server, so it works on fresh documents.
I'd use a high-stakes worked example (verifying clinical claims against the medical literature) since that's where the failure mode bites hardest — but the technique is domain-general.
### Background
I build LLM systems for healthcare and have published peer-reviewed work on constraining models in high-stakes settings; this citation-verification pattern is one I've shipped in production. Happy to bring it into a vendor-neutral, broadly useful guide.
### Before I start
Is a guide on this welcome, and do you have a preferred home for it (`capabilities/` vs `patterns/`)? I'll follow CONTRIBUTING (uv environment, pre-commit hooks, author registry) and keep it to one self-contained notebook.
貢獻指南
評估
這個 Issue 還沒有評估資料。