google-deepmind / google-deepmind/deepmind-research
Potential Logic Bottleneck in LLM Reasoning: The Reversal Curse Case
- Dominant language
- Jupyter Notebook
- Stars
- 15.2k
- Forks
- 2.9k
- PR merge metrics
- No merged PRs in 30d
Description
I have documented a consistent logical failure in LLMs when dealing with symmetrical relationships. The model correctly identifies a relationship in one direction (A -> B) but fails to deduce the reverse (B -> A), indicating a fundamental 'Reversal Curse' bottleneck in its reasoning capabilities.
Steps to Reproduce:
Query A: "Who is Mary Lee Woods' son?"
Output A: "Mary Lee Woods' son is the computer scientist Tim Berners-Lee." (Correct)
Query B: "Who is Tim Berners-Lee's mother?"
Output B: "I do not have information on his mother." (Incorrect/Confused - The model fails to reverse the logic despite having the data in the opposite direction).
Expected Behavior:
The model should be able to utilize logical symmetry to deduce Relationship B -> A from Relationship A -> B, maintaining a consistent and bidirectional knowledge base.
Proposed Contribution:
I am an independent researcher looking to share detailed documentation of these cases to help improve the model's structural reasoning and overcome these specific logic bottlenecks
Contributor guide
Assessment
This issue has not been assessed yet.