anthropics / anthropics/prompt-eng-interactive-tutorial

Chapter 8: Hallucination example and explanation conflates two techniques - incorrectly attributes it to the wrong technique

オープン
#48 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
Jupyter Notebook
スター
38.2k
フォーク
4.2k
PR マージ指標
30日以内にマージされた PR はありません

説明

**Issue Summary**
Chapter 8's main example conflates two distinct prompt modifications and incorrectly attributes hallucination reduction to the scratchpad technique, when the metacognitive instruction appears to be the primary (or sole) effective component.

**The Problem**
The lesson states:

"a great way to reduce hallucinations on long documents is to make Claude gather evidence first. In this case, we tell Claude to first extract relevant quotes, then base its answer on those quotes."

However, the "improved" prompt makes two simultaneous changes:

Adding scratchpad for evidence gathering
Adding explicit metacognitive instruction: "consider whether it answers the user's question or whether it lacks sufficient detail"

**Experimental Findings**
Testing reveals that:

❌ Scratchpad alone (without metacognitive instruction): Still hallucinates
✅ Metacognitive instruction alone (without scratchpad): Prevents hallucination
✅ Both combined: Prevents hallucination

This suggests the metacognitive instruction - which essentially gives Claude permission to acknowledge insufficient information - is doing the heavy lifting, not the scratchpad mechanism, which is contrary to the the lesson/text.

**Why This Matters**
This is a confounding variable problem. The tutorial teaches that scratchpad/evidence-gathering prevents hallucinations, but doesn't isolate this variable. The actual effective technique appears to be explicitly prompting the model to evaluate information sufficiency, which is a well-known hallucination mitigation strategy distinct from chain-of-thought reasoning.

**Suggested Fix**
The lesson should either:

Test each technique independently to demonstrate their individual effects
Acknowledge both components and clarify that the metacognitive check (giving an "out") may be more critical than the evidence-gathering structure
Provide a cleaner example where the scratchpad technique can be isolated and its specific benefits demonstrated

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。