AltimateAI / AltimateAI/altimate-code
[Bug] SQL equivalence and compare tools return contradictory results
- 主要語言
- TypeScript
- 星號
- 811
- 分支
- 134
- 平均合併
- 3 天 2 小時
- 30 天內合併 PR
- 50
描述
## Problem
`altimate_core_equivalence` and `altimate_core_compare` give opposite results for the same query pair. One reports "DIFFERENT" while the other reports "IDENTICAL" for semantically equivalent queries that only differ in condition ordering.
This directly erodes user trust in automated analysis.
## Steps to Reproduce
Compare these two queries:
```sql
-- Query A
SELECT id FROM t WHERE a = 1 AND b = 2 ORDER BY c
-- Query B
SELECT id FROM t WHERE b = 2 AND a = 1 ORDER BY c ASC
```
- `altimate_core_equivalence` → reports "DIFFERENT"
- `altimate_core_compare` → reports "IDENTICAL"
The queries are semantically identical (AND is commutative, `ORDER BY c` = `ORDER BY c ASC`).
## Expected Behavior
Both tools should agree. For semantically equivalent queries with reordered conditions:
- Equivalence check should report EQUIVALENT (commutativity of AND)
- Compare should report IDENTICAL or EQUIVALENT
## Investigation Needed
1. **Understand the difference between the two tools** — what is each tool's intended purpose?
- Is `equivalence` doing AST comparison (syntactic) while `compare` does semantic analysis?
- Or vice versa?
2. **Define clear semantics for each** — if one is syntactic and one is semantic, document this clearly and adjust naming
3. **Consider unifying** — having two tools that answer the same question differently is confusing. Options:
- Merge into a single `sql_compare` tool that reports both syntactic and semantic similarity
- Keep both but return structured results: `{ syntactic: "different", semantic: "equivalent", explanation: "..." }`
- Remove the less accurate one
## Impact
- Users who rely on one tool get wrong answers
- The agent compensates by calling both and explaining the contradiction, but this wastes tokens and looks unreliable
- For compliance/audit use cases, contradictory results are unacceptable
## Acceptance Criteria
- [ ] Root cause identified (why the tools disagree)
- [ ] Tools either unified or given clearly distinct purposes with accurate naming
- [ ] Reordered WHERE conditions correctly identified as equivalent
- [ ] `ORDER BY c` and `ORDER BY c ASC` treated as identical
- [ ] No contradictory results for the same input pair
貢獻指南
評估
這個 Issue 還沒有評估資料。