microsoft / microsoft/typespec

Proposal: Add/extend typeSpec emitters for database schemas

Open
#10,334 5 comments 3 reactions 1 assignee Claimed by @markcowl View on GitHub
compiler:core design:needed triaged:core
Dominant language
Java
Stars
5.9k
Forks
394
Avg merge
1d 23h
Merged PRs (30d)
104

Description

# Proposal: Add/extend typeSpec emitters for database schemas

## Summary

Extend TypeSpec's emitter ecosystem — currently covering OpenAPI, JSON Schema, Avro, and Protobuf, etc.. to generate database schemas

| Typespec emitter | Database/format | Status |
| -------------------- | ------------------------------ | -------- |
| `@typespec/spanner` | Google Cloud Spanner DDL | proposed |
| `@typespec/postgres` | PostgreSQL DDL | propose |
| `@typespec/mysql` | MySQL DDL | proposed |
| `@typespec/bigquery` | Google BigQuery DDL | proposed |
| `@typespec/oracle` | Oracle Database DDL | proposed |
| `@typespec/iceberg` | Apache Iceberg DDL (Spark SQL) | proposed |
| `@typespec/parquet` | Apache Parquet schema (JSON) | proposed |

## Motivation

TypeSpec is a language for defining logical shape of a data, designed as a single source of truth that can generate API specs, JSON Schema, Protobuf schema, etc... but it currently does not support database schemas.

Teams define models like `User` or `Order` in TypeSpec then has to hand-write separate DDL for the same entities for their databases. These creates schema drift silently. Extending TypeSpec to emit database schema completes the "define once, emit many".

## Design Principles

### 1. Maximize reuse of built-in TypeSpec features

Most database concepts already exist in TypeSpec:

| TypeSpec Feature | Database Concept |
| -------------------------------------------------------- | -------------------------- |
| `@maxLength(N)` on `string` | `STRING(N)` / `VARCHAR(N)` |
| No`@maxLength` on `string` | `STRING(MAX)` / `TEXT` |
| Optional`?` | Nullable column |
| Non-optional | `NOT NULL` |
| `@doc(text)` | SQL comment |
| Built-in scalars (`int64`, `float64`, `plainDate`, etc.) | Target column types |
| `T[]` array | `ARRAY` / JSON / LIST |

### 2. Minimal new decorators

Following shows the new decorators that TypeSpec currently dont support:

| Decorator | Purpose |
| --------------------------- | ------------------------------------------------------------------- |
| `@primaryKey(order?)` | Composite primary keys |
| `@unique` | Unique constraint |
| `@references(Model, opts)` | Foreign keys |
| `@index(opts)` | Secondary indexes |
| `@schema("name")` | Scope models to a database (also controls which models are emitted) |
| `@columnOption(key, value)` | Generic extensible per-column metadata |

### 3. Target-specific behavior via scalars and emitter options

When a database has unique types, use custom scalars (not decorators):

```typespec
// Types that carry behavior — no decorator needed
scalar commitTimestamp extends utcDateTime; // Spanner: allow_commit_timestamp
scalar serial extends int64; // Postgres: SERIAL
scalar autoIncrement extends int64; // MySQL: AUTO_INCREMENT
scalar uuid extends string; // Iceberg/Parquet: UUID
```

Infrastructure concerns (partitioning, engines, change streams) go in emitter options.

### 4. Same emitter architecture as Protobuf

Each emitter is self-contained — decorators and emitter in one package. This follows how `@typespec/protobuf` and `@typespec/json-schema` work.

If shared decorators are needed later, they can be extracted into `@typespec/db` (like the `@typespec/openapi` extraction).

## Example: One Schema, Multiple Targets

### TypeSpec Input

```typespec
import "@typespec/spanner";
using TypeSpec.Spanner;

@schema("music_db")
namespace MusicDB;

model Singers {
@primaryKey singerId: int64;
@maxLength(1024) firstName: string;
@maxLength(1024) lastName: string;
birthDate?: plainDate;
}

@interleave(Singers, "cascade")
@index({ columns: ["albumTitle"], name: "AlbumsByTitle" })
model Albums {
@primaryKey(1) singerId: int64;
@primaryKey(2) albumId: int64;
albumTitle?: string;
lastUpdated?: commitTimestamp;
}
```

```### PostgreSQL Output (same models, different emitter)

```sql
CREATE SCHEMA IF NOT EXISTS music_db;

CREATE TABLE music_db.singers (
singer_id BIGINT NOT NULL,
first_name VARCHAR(1024) NOT NULL,
last_name VARCHAR(1024) NOT NULL,
birth_date DATE,
PRIMARY KEY (singer_id)
);

CREATE TABLE music_db.albums (
singer_id BIGINT NOT NULL,
album_id BIGINT NOT NULL,
album_title TEXT,
last_updated TIMESTAMPTZ,
PRIMARY KEY (singer_id, album_id)
);

CREATE INDEX albums_by_title ON music_db.albums(album_title);
```
### Parquet Output (same models, JSON schema)

```json
{
"type": "message",
"name": "singers",
"fields": [
{ "name": "singer_id", "repetition": "REQUIRED", "physical_type": "INT64" },
{ "name": "first_name", "repetition": "REQUIRED", "physical_type": "BYTE_ARRAY", "logical_type": "STRING" },
{ "name": "birth_date", "repetition": "OPTIONAL", "physical_type": "INT32", "logical_type": "DATE" }
]
}
```
## Type Mapping Across All Targets

| TypeSpec | Spanner | Postgres | MySQL | BigQuery | Oracle | Iceberg | Parquet |
| ------------------------ | ----------- | ----------- | ---------- | --------- | -------------- | --------- | ---------- |
| `boolean` | BOOL | BOOLEAN | TINYINT(1) | BOOL | NUMBER(1) | BOOLEAN | BOOLEAN |
| `int32` | INT64 | INTEGER | INT | INT64 | NUMBER(10) | INT | INT32 |
| `int64` | INT64 | BIGINT | BIGINT | INT64 | NUMBER(19) | LONG | INT64 |
| `float32` | FLOAT32 | REAL | FLOAT | FLOAT64 | BINARY_FLOAT | FLOAT | FLOAT |
| `float64` | FLOAT64 | DOUBLE PREC | DOUBLE | FLOAT64 | BINARY_DOUBLE | DOUBLE | DOUBLE |
| `string` | STRING(MAX) | TEXT | TEXT | STRING | VARCHAR2(4000) | STRING | STRING |
| `string`+`@maxLength(N)` | STRING(N) | VARCHAR(N) | VARCHAR(N) | STRING(N) | VARCHAR2(N) | STRING | STRING |
| `bytes` | BYTES(MAX) | BYTEA | BLOB | BYTES | RAW(2000) | BINARY | BYTE_ARRAY |
| `plainDate` | DATE | DATE | DATE | DATE | DATE | DATE | DATE |
| `utcDateTime` | TIMESTAMP | TIMESTAMPTZ | DATETIME | TIMESTAMP | TIMESTAMP | TIMESTAMP | TIMESTAMP |
| `decimal` | NUMERIC | NUMERIC | DECIMAL | NUMERIC | NUMBER | DECIMAL | DECIMAL |
| `T[]` | ARRAY\ | T[] | JSON | ARRAY\ | _(warn)_ | ARRAY\ | LIST |

## Scope

**In scope:** OLTP databases (Spanner, PostgreSQL, MySQL, Oracle), analytics (BigQuery), and data formats (Iceberg, Parquet).

**Out of scope:** Schema migrations, ORM/query generation, views/stored procedures, non-relational databases (BigTable, MongoDB, DynamoDB).

### Checklist

- [x] Follow our [Code of Conduct](https://github.com/microsoft/typespec/blob/main/CODE_OF_CONDUCT.md)
- [x] Read the [docs](https://typespec.io/docs/).
- [x] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.