dair-ai / dair-ai/Prompt-Engineering-Guide

add technique: 'Self-Discover: Large Language Models Self-Compose Reasoning Structures'

Open
#448 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
MDX
Stars
78.3k
Forks
8.6k
PR merge metrics
No merged PRs in 30d

Description

https://arxiv.org/abs/2402.03620

> Self-Discover: Large Language Models Self-Compose Reasoning Structures
>
> We introduce SELF-DISCOVER, a general framework for LLMs to self-discover the task-intrinsic reasoning structures to tackle complex reasoning problems that are challenging for typical prompting methods. Core to the framework is a self-discovery process where LLMs select multiple atomic reasoning modules such as critical thinking and step-by-step thinking, and compose them into an explicit reasoning structure for LLMs to follow during decoding. SELF-DISCOVER substantially improves GPT-4 and PaLM 2's performance on challenging reasoning benchmarks such as BigBench-Hard, grounded agent reasoning, and MATH, by as much as 32% compared to Chain of Thought (CoT). Furthermore, SELF-DISCOVER outperforms inference-intensive methods such as CoT-Self-Consistency by more than 20%, while requiring 10-40x fewer inference compute. Finally, we show that the self-discovered reasoning structures are universally applicable across model families: from PaLM 2-L to GPT-4, and from GPT-4 to Llama2, and share commonalities with human reasoning patterns.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the linked Self-Discover paper and then inspect the repository's existing technique entries to determine where this addition belongs and what format it should follow. Done means the guide includes the Self-Discover technique with content grounded in the cited paper.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.