anitnilay20 / anitnilay20/thoth

File query: filtering (WHERE clause via DuckDB)

Abierto
#53 1 comentario 0 reacciones 0 asignados Ver en GitHub
enhancement feature performance priority:medium size:large
Lenguaje dominante
Rust
Estrellas
70
Forks
5
Merge medio
10 h 33 min
PR fusionados (30 d)
2

Descripción

## Context

Part of the DuckDB file query engine epic (#147). Depends on #148 (FileQueryEngine) and #149 (query editor UI). The original implementation plan proposed a custom parser and Rayon evaluator — with DuckDB this is replaced entirely by standard SQL `WHERE`.

## What this covers

Exposing DuckDB's `WHERE` clause through the query editor for structured filtering of open files.

### Examples (typed into the query editor)

```sql
SELECT * FROM data WHERE response_time > 500
SELECT * FROM data WHERE status_code != 200 AND host = 'prod'
SELECT * FROM data WHERE price < 100 AND stock > 0
SELECT * FROM data WHERE timestamp > '2024-01-01' AND error_level = 'critical'
SELECT * FROM data WHERE user.age >= 18
SELECT * FROM data WHERE deleted_at IS NULL
```

DuckDB handles type inference, nested field access via dot notation (with `json_extract` under the hood for JSON), NULL semantics, operator precedence, and parenthesised grouping — no custom parser needed.

## What needs building

This ticket is **UI/UX only** — the engine work is in #148, the editor widget in #149. The work here is:

- Validate that DuckDB's JSON path handling covers nested fields from plugin-loaded JSON records (e.g. `user.age`, `items[0].price`)
- Handle the case where `data` schema has dynamic / heterogeneous columns (wide sparse tables from heterogeneous JSON) — surface schema inference errors clearly
- Test filtering across all three ingestion paths: native JSON, native CSV, plugin-ingested

## Acceptance criteria

- [ ] `WHERE` with comparison operators (`>`, `<`, `>=`, `<=`, `!=`, `=`) works on numeric and string columns
- [ ] Logical operators (`AND`, `OR`) with parentheses work correctly
- [ ] Nested field access (`user.age`, `items.length`) works for JSON sources
- [ ] `IS NULL` / `IS NOT NULL` work
- [ ] Type mismatch errors shown inline in the query editor (not a crash)
- [ ] Works for native JSON, native CSV, and plugin-ingested formats

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.