[Research] Calibration context: Should negotiation carry empirical track records?
- Dominant language
- Typst
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
## Research Question
INTENT proposes that each message carry **calibration signatures**—empirical accuracy maps showing how well the sender's confidence correlates with reality.
Is this necessary, sufficient, or overkill for achieving alignment?
## The Problem
Thinker says: "I'm 90% confident quicksort is optimal"
But Thinker's calibration shows:
- In algorithm design: 0.9 confidence → 0.74 actual accuracy
- In systems programming: 0.9 confidence → 0.91 actual accuracy
Should Assembler:
1. Treat as 0.9 (take confidence at face value)?
2. Treat as 0.74 (apply calibration correction)?
3. Request clarification (use calibration to trigger ASK)?
## Hypotheses
1. **Calibration is essential**: Without it, confidence is meaningless across model boundaries
2. **Calibration is advisory**: Receivers may apply it, ignore it, or use it to decide trust
3. **Calibration is temporary**: Only needed until models achieve universal calibration (unlikely)
## Research Direction
- Survey current LLM calibration across domains
- Design protocol for calibration exchange and negotiation
- Determine if calibration should be:
- Mandatory (protocol enforces it)
- Optional (participants decide)
- Emergent (learned from interaction history)
---
*Research thread: Confidence calibration in multi-model negotiation*
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.