clockworklabs / clockworklabs/SpacetimeDB

Add AI Capabilities to SpacetimeDB

Open
#4,910 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
25.2k
Forks
1.1k
Avg merge
2d 7h
Merged PRs (30d)
46

Description

This issue tracks reimplementing the work from stale PR #4589, which has been closed because it is too far out of date to merge directly.

## Original pull request
- PR #4589: [[Draft] Add AI Capabilities to SpacetimeDB](https://github.com/clockworklabs/SpacetimeDB/pull/4589)
- Original author: @definenoob

## What the original PR was trying to do
With this PR, STDB becomes the first AI-enabled database. **Seeking feedback from CWL on this**.

Summary

  • Adds native ONNX inference as a host-side capability exposed to WASM modules via the SpacetimeDB ABI (spacetime_10.5)
  • Models are loaded by name from {server_data_dir}/models/{name}.onnx on the host filesystem and cached after first use — model bytes never enter WASM memory, only input/output
    tensor data crosses the boundary
  • Uses tract-onnx (pure Rust) for inference, no C++ dependencies or WASI required

Design

Single ABI call (spacetime_10.5):

  • onnx_run(name_ptr, name_len, input_ptr, input_len, out) -> errno — loads/caches model by name, runs inference, returns output tensors via BytesSource

Crate changes:

  • spacetimedb-lib — shared Tensor type (Vec shape + Vec data) with BSATN serialization
  • spacetimedb-primitives — ONNX_ERROR errno code
  • spacetimedb-bindings-sys — raw ABI declaration + safe wrapper
  • spacetimedb-bindings — high-level OnnxClient with single run(name, inputs) method, exposed via ReducerContext::onnx and ProcedureContext::onnx
  • spacetimedb-core — host-side OnnxModel using tract-onnx, per-instance model cache (HashMap<String, OnnxModel>) in WasmInstanceEnv, path traversal validation, models_dir wired
    from HostController.data_dir

Usage from a module:

let input = vec![Tensor { shape: vec![1, 10], data: vec![0.0; 10] }];
let output = ctx.onnx.run("bot_brain", &input)?;

Test plan

  • Place a .onnx model file in {data_dir}/models/ and verify ctx.onnx.run("model_name", &input) succeeds
  • Verify inference produces correct output tensors for known model inputs
  • Verify second call to same model uses cache (no re-parse)
  • Verify invalid model names (path traversal, empty) return errors
  • Verify loading a nonexistent model returns a clear error
  • Verify compilation on both native and wasm32-unknown-unknown targets

The onnx feature flag works across all three crates:

  • spacetimedb (bindings): features = ["onnx"] — enables ctx.onnx.run(...) API
  • spacetimedb-bindings-sys: features = ["onnx"] — enables raw ABI declaration
  • spacetimedb-core: features = ["onnx"] — enables host-side inference + tract-onnx dependency

Without the feature, tract-onnx isn't compiled, no ONNX code exists in the binary, and the ABI function isn't registered.

This is ACID-compliant AI inference.

run the model inside the reducer to decide what to write. The inference result feeds directly into the transactional state change — no round-trip to an external service, no eventual consistency, just one atomic unit of "think and act."

## Closure context
- Closed from the `close-stale-pr-create-issue` stale-PR sweep.

## Reimplementation notes
- Reimplement this work on top of current `master` in a new PR.
- Keep the original PR linked as historical context and as a source of useful implementation ideas where still relevant.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing closed PR #4589, then compare its proposed changes with the current spacetimedb-lib, spacetimedb-primitives, spacetimedb-bindings-sys, spacetimedb-bindings, and spacetimedb-core crates. Use the listed test plan as the acceptance checklist, including model loading, inference output, caching, invalid names, missing models, and native and wasm32 compilation.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
ai, backend, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.