AltimateAI / AltimateAI/altimate-code

18 tool issues found across SQL Tools, Core Tools, Schema Tools, and Lineage

Aperta
#422 1 commento 0 reazioni 1 assegnatario Rivendicata da @mhallida Vedi su GitHub
bug from-cli user-feedback
Lingua principale
TypeScript
Stelle
811
Fork
134
Merge medio
3g 2h
PR unite (30g)
50

Descrizione

## Summary

Ran a comprehensive docs-as-spec test against every tool documented on [docs.altimate.sh](https://docs.altimate.sh). Tested 46 tools using the exact examples from the docs plus edge cases. **34 passed, 18 issues found.**

A full detailed report with reproduction steps, expected vs actual output, impact analysis, and suggested fixes for every issue is attached below.

---

## Priority Breakdown

### P1 — Critical (3 issues)
- **`sql_diff`** — Reports clearly different queries as "identical" (docs example fails)
- **`altimate_core_compare`** — Same comparison bug, likely shared root cause with `sql_diff`
- **`sql_fix`** — Returns the original unfixed SQL instead of applying the fix

### P2 — Medium (10 issues)
- **`sql_rewrite`** — "No rewritable anti-patterns" on a textbook `YEAR()` non-sargable predicate
- **`sql_optimize`** — Detects anti-patterns but returns unmodified SQL (acts as second `sql_analyze`)
- **`sql_translate`** — 5 Snowflake→BigQuery constructs fail: `TRY_TO_NUMBER`, `ARRAY_AGG WITHIN GROUP`, `IFF`→`IF`, `QUALIFY` (invalid subquery wrapping), `FLATTEN`→`UNNEST`
- **`altimate_core_fix`** — Can't fix `SELCT`→`SELECT` keyword typos
- **`altimate_core_correct`** — Same fix failure + `[object Object]` serialization bug in output
- **`altimate_core_equivalence`** — `= 'x'` vs `IN ('x')` reported as non-equivalent
- **`altimate_core_track_lineage`** — Returns empty for multi-query `CREATE TABLE AS SELECT` lineage
- **`altimate_core_migration`** — Misses FLOAT→DECIMAL type narrowing (schema_diff catches it)
- **`altimate_core_classify_pii`** — Misses `credit_card_number` column as PII (0 confidence)
- **`schema_diff`** (SQL-level) — Returns empty diff on clearly different DDL ("? → ? columns")

### P3 — Low (4 issues)
- **`sql_format`** — Outputs single-line SQL instead of multi-line indented format
- **`altimate_core_semantics`** — Returns empty for implicit cross joins
- **`sql_analyze`** — 2 of 19 documented rules don't fire: `WINDOW_WITHOUT_PARTITION`, `LARGE_IN_LIST`
- **`altimate_core_policy`** — Rejects valid DML before checking policies

---

## What Works Well (28+ tools passing)

`sql_analyze`, `altimate_core_check`, `altimate_core_column_lineage` (simple queries), `altimate_core_classify_pii` (email/phone/SSN/IP/name/DOB), `altimate_core_query_pii`, `altimate_core_import_ddl`, `altimate_core_export_ddl`, `altimate_core_testgen` (24 test cases generated), `altimate_core_complete`, `altimate_core_prune_schema`, `altimate_core_optimize_context`, `altimate_core_resolve_term`, `altimate_core_introspection_sql`, `altimate_core_fingerprint`, `altimate_core_grade`, `altimate_core_schema_diff` (via schema_context), `lineage_check`, all Memory tools, all Training tools, all Warehouse tools.

---

## Full Report

The complete report with all 18 tickets, reproduction steps, expected vs actual outputs, and suggested fixes is in the attached file: `altimate-code-test-tickets.md`

Each ticket includes:
- Priority and component
- Exact input used (matching docs examples where possible)
- Expected output (per docs)
- Actual output (verbatim)
- Impact analysis
- Suggested fix

### Proposed Approach
Fix all 18 issues in a single PR rather than 18 separate ones, since many share root causes:
- `sql_diff` + `altimate_core_compare` likely share comparison logic
- `sql_fix` + `altimate_core_fix` + `altimate_core_correct` all relate to the fix/correct pipeline
- `sql_optimize` + `sql_rewrite` both fail at the AST rewrite stage

---

### Metadata

| Field | Value |
|-------|-------|
| CLI Version | 0.5.7 |
| Platform | darwin |
| Architecture | arm64 |
| OS Release | 25.3.0 |
| Category | bug |
| Working Directory | AI-Tools |
| Session ID | ses_2e4130cb2ffeCqOgttOduwkspe |

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.