Conformance note: sqllogictest and SQLite's TCL suite are very different bars
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 279
- Forks
- 25
- PR merge metrics
- No merged PRs in 30d
Description
Hi — really enjoyed the agent-swarm model-economics post and minisqlite as its artifact. I've been building a parallel thing from a different direction: a pure-Rust SQLite-compatible engine driven by agent orchestration. I am opening this as a constructive conformance note rather than a bug report — feel free to close if it's not useful.
The observation
The post's headline is 100% on the held-out sqllogictest suite. That's a real milestone — but from a lot of time in the SQLite-conformance trenches, I'd gently push back on treating it as "SQLite parity," because sqllogictest and SQLite's own TCL test suite are wildly different difficulty bars, and the gap is easy to underestimate.
- sqllogictest almost entirely asserts result rows for well-formed queries.
- SQLite's TCL suite asserts SQLite's documented behavior: exact error-message text,
PRAGMAoutput,sqlite_master/schema details, trigger timing and statement legality, DML edge semantics, the on-disk file format, the C-API, collations, and the "documentation-evidence" tests (e_select,e_insert,e_update, …) that verify each individually-testable statement in the SQL language docs.
The second surface is categorically larger and stricter, and it's where the genuine long tail of conformance work lives.
Concrete flavor
A representative sample of things the TCL suite pins down that a result-row harness never exercises:
- Emitting SQLite's exact diagnostics, e.g.
aggregate functions are not allowed in the GROUP BY clause, ora JOIN clause is required before ON(not just an error, the right error text). - Rejecting
ORDER BY/LIMITinside a trigger body with SQLite's precisenear "..."wording. REINDEXbehavior observed through a user-defined collation.- Integer-affinity edge cases on
LIMIT/OFFSET, HAVING-only aggregates that must not leak into the projection, etc.
Hard to say which is the better benchmark but "cloning sqlite" is much harder than "passing sqllogictest".
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No repository file, test target, or entry point is named. Start by comparing the sqllogictest harness with SQLite's TCL test suite and its documented conformance cases; done is not defined because the note proposes no specific change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sqlite
- Domain
- databases, testing-qa
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100