vercel-labs / vercel-labs/native

Design: persistence for TS cores without eject (SQL in-process or file KV?)

Open
#309 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Zig
Stars
7.7k
Forks
314
Avg merge
5h
Merged PRs (30d)
13

Description

Many real apps need persistence beyond a flat blob, because the logic is relational: entities reference each other, queries aggregate, and writes touch several tables at once. A whole-model file with writeFile/readFile can carry that, but then the app owns a serialization format and every query as a hand-written scan. I've been mapping what the TS core path actually offers, and I want to check the intended direction.

What I found so far:

  • Cmd.persist() compiles but no host implements it yet (NS1028); the ts-core skill points at Cmd.writeFile/Cmd.readFile as the persistence pattern.
  • Cmd.request/Cmd.host with a custom name comes back rejected in the default wiring, because the generated TS wiring binds no host services. I confirmed in ts_core_main.zig and effects.zig that unbound host calls get an err: "rejected" result.
  • That leaves, as I see it: a whole-model file KV with a hand-rolled format, spawning a system sqlite3 CLI per query, or ejecting to bind a Zig host service with SQLite linked in-process. npm-based bindings (libsql etc.) are ruled out by the subset.

Questions:

  1. Is in-process storage for TS cores on the roadmap (a builtin KV or SQL host service), or is the file KV pattern the intended answer for now?
  2. If the file KV pattern is it: any guidance on the 1 MiB per-file ceiling? And is NATIVE_SDK_APP_DATA_DIR supposed to be set by the runner? The voice-memo example reads it through envMsgs, but I couldn't find anywhere in the CLI that injects it, so I'm deriving the data dir from HOME/XDG myself.
  3. I'd rather not spawn a subprocess per query, partly for latency and partly because it pins the app to whatever sqlite version the user has. Is that pattern something you'd discourage?

I think relational data is common enough that the answer shouldn't be "eject and write Zig." Curious what you'd recommend.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading ts_core_main.zig and effects.zig, then trace the Cmd.persist(), Cmd.writeFile(), Cmd.readFile(), Cmd.request(), and Cmd.host() paths described in the issue. Done would be a maintainer-confirmed persistence direction for TS cores, including guidance on the file-size ceiling, NATIVE_SDK_APP_DATA_DIR, and subprocess-based SQLite.

Written by the indexing model from the issue text.

Assessment

Tech stack
sqlite, typescript, zig
Domain
databases, desktop
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.