AltimateAI / AltimateAI/altimate-code

18 tool issues found across SQL Tools, Core Tools, Schema Tools, and Lineage

Abierto
#422 1 comentario 0 reacciones 1 asignado Reclamado por @mhallida Ver en GitHub
bug from-cli user-feedback
Lenguaje dominante
TypeScript
Estrellas
811
Forks
134
Merge medio
3 d 2 h
PR fusionados (30 d)
50

Descripción

## Summary

Ran a comprehensive docs-as-spec test against every tool documented on [docs.altimate.sh](https://docs.altimate.sh). Tested 46 tools using the exact examples from the docs plus edge cases. **34 passed, 18 issues found.**

A full detailed report with reproduction steps, expected vs actual output, impact analysis, and suggested fixes for every issue is attached below.

---

## Priority Breakdown

### P1 — Critical (3 issues)
- **`sql_diff`** — Reports clearly different queries as "identical" (docs example fails)
- **`altimate_core_compare`** — Same comparison bug, likely shared root cause with `sql_diff`
- **`sql_fix`** — Returns the original unfixed SQL instead of applying the fix

### P2 — Medium (10 issues)
- **`sql_rewrite`** — "No rewritable anti-patterns" on a textbook `YEAR()` non-sargable predicate
- **`sql_optimize`** — Detects anti-patterns but returns unmodified SQL (acts as second `sql_analyze`)
- **`sql_translate`** — 5 Snowflake→BigQuery constructs fail: `TRY_TO_NUMBER`, `ARRAY_AGG WITHIN GROUP`, `IFF`→`IF`, `QUALIFY` (invalid subquery wrapping), `FLATTEN`→`UNNEST`
- **`altimate_core_fix`** — Can't fix `SELCT`→`SELECT` keyword typos
- **`altimate_core_correct`** — Same fix failure + `[object Object]` serialization bug in output
- **`altimate_core_equivalence`** — `= 'x'` vs `IN ('x')` reported as non-equivalent
- **`altimate_core_track_lineage`** — Returns empty for multi-query `CREATE TABLE AS SELECT` lineage
- **`altimate_core_migration`** — Misses FLOAT→DECIMAL type narrowing (schema_diff catches it)
- **`altimate_core_classify_pii`** — Misses `credit_card_number` column as PII (0 confidence)
- **`schema_diff`** (SQL-level) — Returns empty diff on clearly different DDL ("? → ? columns")

### P3 — Low (4 issues)
- **`sql_format`** — Outputs single-line SQL instead of multi-line indented format
- **`altimate_core_semantics`** — Returns empty for implicit cross joins
- **`sql_analyze`** — 2 of 19 documented rules don't fire: `WINDOW_WITHOUT_PARTITION`, `LARGE_IN_LIST`
- **`altimate_core_policy`** — Rejects valid DML before checking policies

---

## What Works Well (28+ tools passing)

`sql_analyze`, `altimate_core_check`, `altimate_core_column_lineage` (simple queries), `altimate_core_classify_pii` (email/phone/SSN/IP/name/DOB), `altimate_core_query_pii`, `altimate_core_import_ddl`, `altimate_core_export_ddl`, `altimate_core_testgen` (24 test cases generated), `altimate_core_complete`, `altimate_core_prune_schema`, `altimate_core_optimize_context`, `altimate_core_resolve_term`, `altimate_core_introspection_sql`, `altimate_core_fingerprint`, `altimate_core_grade`, `altimate_core_schema_diff` (via schema_context), `lineage_check`, all Memory tools, all Training tools, all Warehouse tools.

---

## Full Report

The complete report with all 18 tickets, reproduction steps, expected vs actual outputs, and suggested fixes is in the attached file: `altimate-code-test-tickets.md`

Each ticket includes:
- Priority and component
- Exact input used (matching docs examples where possible)
- Expected output (per docs)
- Actual output (verbatim)
- Impact analysis
- Suggested fix

### Proposed Approach
Fix all 18 issues in a single PR rather than 18 separate ones, since many share root causes:
- `sql_diff` + `altimate_core_compare` likely share comparison logic
- `sql_fix` + `altimate_core_fix` + `altimate_core_correct` all relate to the fix/correct pipeline
- `sql_optimize` + `sql_rewrite` both fail at the AST rewrite stage

---

### Metadata

| Field | Value |
|-------|-------|
| CLI Version | 0.5.7 |
| Platform | darwin |
| Architecture | arm64 |
| OS Release | 25.3.0 |
| Category | bug |
| Working Directory | AI-Tools |
| Session ID | ses_2e4130cb2ffeCqOgttOduwkspe |

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.