anthropics / anthropics/claude-cookbooks

Proposal: guide on citation-faithfulness evals — catching hallucinated/unsupported citations in grounded generation

未關閉
#682 1 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Jupyter Notebook
星號
52.7k
分支
6.3k
平均合併
25 分鐘
30 天內合併 PR
6

描述

### What I'd like to add

A single self-contained guide on **evaluating citation faithfulness** — verifying that every claim a model makes is actually supported by the source it cites, and catching *hallucinated or misattributed citations* before they reach a user. It's a failure mode nearly every RAG / grounded-generation app hits, but I couldn't find a guide covering it directly.

### Why it's useful (and not a duplicate)

I reviewed the existing evaluation material first, per the contributing guidelines:

- `capabilities/classification/evaluation` — classification accuracy
- `capabilities/knowledge_graph/evaluation` — extraction quality
- `tool_evaluation/` and `capabilities/retrieval_augmented_generation` — RAG / tool patterns

These evaluate *retrieval* and *task accuracy*, but none check whether a generated claim's **cited passage can actually be located in the source** and whether it **supports the claim**. That's the gap this guide fills.

### Rough outline

1. The problem: plausible-but-unsupported citations, and why retrieval metrics miss them.
2. An eval that, per claim: extracts the cited passage → verifies it exists verbatim in the source → uses Claude as judge to grade support level (supports / partial / contradicts / not found).
3. A small labelled dataset + scoring (precision/recall on "unsupported-citation" detection).
4. Hardening: rejecting hallucinated quotes, confidence calibration.
5. *(Optional)* a variant that pulls sources live via an MCP server, so it works on fresh documents.

I'd use a high-stakes worked example (verifying clinical claims against the medical literature) since that's where the failure mode bites hardest — but the technique is domain-general.

### Background

I build LLM systems for healthcare and have published peer-reviewed work on constraining models in high-stakes settings; this citation-verification pattern is one I've shipped in production. Happy to bring it into a vendor-neutral, broadly useful guide.

### Before I start

Is a guide on this welcome, and do you have a preferred home for it (`capabilities/` vs `patterns/`)? I'll follow CONTRIBUTING (uv environment, pre-commit hooks, author registry) and keep it to one self-contained notebook.

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。