Maybe drop the `sql` tagged template

Open
#2,052 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
30/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
nodejs, postgresql, typescript

Research direction

Start by reading query.ts:877-945 and Expr.ts:519-576, then inspect the 23 uses in query.test.ts and the related type, expression, and CTE tests. Review the raw-query documentation sections listed in queries-raw.md. Done would require an agreed design, migrated tests and docs, removal of the old APIs, and a release note; the issue leaves several design choices open.

Written by the indexing model from the issue text.

Description

Dropping the sql tagged template

Exploration of hard-deleting sql / sql.condition / sql.ref / .is and replacing them with
expression literals: a handful of new named expressions plus a { sql: ... } escape hatch.

What exists today

Two parallel ways to build a value expression:

Style Examples Where it lives
Expression literals { coalesce }, { nullIf }, { greatest }, { least }, { case }, { arrayAgg } queries/sql/expressions/
Tagged templates sql`...` , sql.number, ..., sql.condition, sql.ref, expr.is`...` query.ts:877-945, Expr.ts:519-576

The literal side is the newer, better-developed one. It has:

  • Per-operand type inference (ExpressionValue, CoalesceValue), including LEFT-join nullability.
  • Real codecs (chooseExpressionCodec), so { coalesce: [b.id, "b:9"] } encodes a tagged Book id.
  • Key validation (checkKeys) that rejects typos instead of ignoring them.
  • Operand pruning that drops unused joins.

The template side has none of that. TemplateExpr.outputType is undefined, so sql<R> is an
unchecked assertion, and the docs already have to carve out an exception for it:

Outputs from sql<R> and sql.ref have unknown codecs and are rejected [in set operations], even
if both operands reuse the same expression... A generic annotation or a cast inside raw SQL does
not declare a codec.

So this is not only a readability cleanup. Folding raw SQL into the literal grammar is the only place
we can ask for a declared result type, which closes that hole.

The { sql: ... } escape hatch

Shape
interface SqlInput {
  /** Raw SQL with `?` placeholders, one per `args` entry. `??` is a literal `?`. */
  readonly sql: string;
  /** Expressions, conditions, or bound values, in placeholder order. */
  readonly args?: readonly unknown[];
  /** The result codec: a type token, or "decode like this expression". */
  readonly type?: TypeToken | ExprLike<unknown>;
  /** Defaults to true; `false` asserts the expression is NOT NULL. */
  readonly nullable?: boolean;
}

? is chosen because it is already the binding syntax of SqlFragment, so the user's string and
Joist's internal fragments speak the same language and rendering is a splice, not a translation.

Each ? consumes one args entry and resolves exactly like a ${} does today
(interpolationToSql): an Expr renders its aliased SQL, a condition renders its predicate,
anything else becomes a binding. Mismatched counts throw, which is strictly better than the template
form, where arity is structurally impossible to get wrong and therefore never checked.

Before / after
// Value expression
sql.number`${a.age} + ${2}`               →  { sql: "? + ?", args: [a.age, 2], type: "int" }
sql.string`upper(${a.first_name})`        →  { sql: "upper(?)", args: [a.first_name], type: "text" }
sql.stringOrNull`${a.age}::text`          →  { sql: "?::text", args: [a.age], type: "text" }

// Condition
sql.condition`${a.age} > ${18}`           →  { sql: "? > ?", args: [a.age, 18] }
sql.condition`true`                       →  { sql: "true" }

// Unmodeled column
sql.ref(a, "ts_search")                   →  ref(a, "ts_search")
sql.condition`${sql.ref(a, "ts_search")} @@ plainto_tsquery(${words})`
                                          →  { sql: "? @@ plainto_tsquery(?)", args: [ref(a, "ts_search"), words] }

// Predicate suffix
a.age.is`BETWEEN ${min} AND ${max}`       →  a.age.between(min, max)         // already modeled
ref(ts, "range").is`@> ${asOf}::timestamptz`
                                          →  { sql: "? @> ?::timestamptz", args: [ref(ts, "range"), asOf] }
Conditions and expressions share the key

QueryCondition (query.ts:116) is already a literal union — { and }, { or }, { exists },
{ notExists } — so { sql, args } joins it as one more member. In a condition position it renders
as a predicate; in a value position it renders as an expression and type supplies the codec. That
removes the sql / sql.condition split entirely: one key, and position decides. where already
accepts ExprLike<boolean> alongside SqlCondition, so the two readings never conflict.

Declared result types

type is the real win. A token maps to a TypeInfo and therefore to both the TS result type and a
runtime codec:

const tokens = {
  int: { dbType: "int4", domain: Number, arrayElementSafe: true },
  bigint: { dbType: "int8", domain: BigInt },
  numeric: { dbType: "numeric", domain: Number, arrayElementSafe: true },
  text: { dbType: "text", domain: String, arrayElementSafe: true },
  boolean: { dbType: "bool", domain: Boolean, arrayElementSafe: true },
  timestamptz: { dbType: "timestamptz", domain: Date },
  // ...plus `${token}[]` for arrays
} as const satisfies Record<string, TypeInfo>;

type: a.age is the other form: "whatever that expression decodes to." That covers tagged ids and
custom types without inventing a token for every mapper, i.e.
{ sql: "greatest(?, ?)", args: [b.id, m.id], type: b.id } returns a BookId.

Both forms make raw SQL a first-class set-operation operand, which sql<R> can never be. Omitting
type keeps today's unknown-codec behavior, so the escape hatch stays cheap for one-off selects.

The named expressions to add first

Working backwards from every sql use in the repo and docs, these cover all but the genuinely
exotic ones:

Literal Replaces Notes
{ concat: [...] } sql`${a.first_name} || ${"!"}` Text codec, null-propagating like ||
{ upper: x }, { lower: x }, { trim: x } sql`upper(${x})` Preserves the operand's string codec
{ cast: [x, "text"] } sql`${x}::text` Declares the result codec, unlike a raw ::
{ add: [...] }, { subtract: [a, b] }, { multiply: [...] }, { divide: [a, b] } sql`${a.age} + ${2}` The main readability judgment call — see below
{ abs: x }, { round: [x, 2] }, { floor: x }, { ceil: x } Cheap, numeric codec from the operand
{ bool: condition } sql.boolean`${a.age.gte(18)}` A condition used as a selected value
{ now: true } sql`now()` timestamptz

Deliberately not in the first pass: window functions (row_number() OVER (...)) and FILTER on
unmodeled aggregates. Aggregates already take filter, and windows want their own
{ rowNumber: { partitionBy, orderBy } } design rather than being rushed in alongside this.

{ add: [a.age, 2] } versus sql`${a.age} + 2` is the one place where the object form is
arguably worse, and it is also the single most common use of the template in the test suite. Two
outs: accept the verbosity (it is rarer in real queries than in tests), or keep operator sugar as
methods — a.age.plus(2) — which is more in keeping with .coalesce(fallback) and .sum() already
being methods on columns. My preference is the methods; they compose left-to-right and avoid an
{ add } literal that reads like Lisp.

What gets deleted

  • query.ts:877-945sql and its 12 sql.* type shorthands, sql.condition, sql.ref.
  • query.ts:2567typedSql.
  • Expr.ts:537-565TemplateExpr, replaced by a RawExpr over (string, args, type).
  • Expr.ts:88,300-303 — the .is tagged template on BaseExpr.
  • Tables.ts:295,315,584,646.raw(exp, bindings), the em.find-era escape hatch. It has zero
    uses outside the type tests and is exactly { sql } with the receiver spliced in.
  • index.ts:236 — the sql export. This is a breaking public API change, so it wants a release note
    and probably a short blog post in the same vein as the pruning one.

RefExpr stays; only its accessor moves from sql.ref to a top-level ref export. It takes a
source plus a string, so it is a function, not an expression literal — { ref: [a, "ts_search"] }
would just be a worse-typed spelling of the same thing.

interpolationToSql stays as-is and becomes the per-? resolver.

Call sites to migrate

23 in query.test.ts, 3 each in query.types.test.ts and query.expressions.test.ts, 1 in
query.ctes.test.ts, and the queries-raw.md sections at lines 147, 220, 255, 562, and 920-954.
(benchmark.ts's sql is postgres.js and unaffected.) Roughly half of those become a named
expression rather than { sql }, which is a decent signal that the escape hatch will be rare.

Costs worth naming

  1. Editor SQL highlighting. sql`...` gets syntax highlighting and Prettier's SQL plugin in
    most setups; { sql: "..." } does not. A /* sql */ "..." comment hint restores it in VS Code,
    and the docs can show that, but it is a real regression for long raw fragments.
  2. The jsonb ? operators. ?, ?|, and ?& collide with the placeholder. ?? escaping plus
    a placeholder/arg arity check makes this loud instead of silent, and modeled pathExists /
    pathIsTrue already cover the common jsonb cases. Still, it is the sharpest edge in the design.
  3. Top-level select ambiguity. select: { sql: "count(*)" } names a column sql holding a
    bound string, exactly as select: { coalesce: a.first_name } names a column coalesce. Correct
    and consistent, but sql is a more tempting key name than coalesce, so it needs a doc callout.
  4. Multi-line raw SQL reads worse as a quoted string than as a template literal. Template
    literals are still available for the string itself — { sql: \...`, args: [...] }` — so this is
    only about the interpolation, not the line breaks.

Suggested sequencing

  1. Add { sql } (expression + condition), ref(), and the type tokens. Keep sql working.
  2. Add the named expressions and the arithmetic methods.
  3. Migrate tests and docs; measure how many uses actually need { sql }.
  4. Delete sql, sql.*, .is, and .raw; write the release note.

Steps 1-2 are additive and independently useful, so the decision to hard-delete can be made after
step 3, with the migration diff in hand rather than predicted.

Dominant language
TypeScript
Stars
385
Forks
25
Avg merge
7h 33m
Merged PRs (30d)
52

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from joist-orm/joist-orm

All issues in joist-orm/joist-orm

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.