pingcap / pingcap/tidb

feature: in-DB model serving and LLM builtins

Open
#66,599 0 comments 2 reactions 0 assignees View on GitHub
ai-native type/feature-request
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Feature Request

**Is your feature request related to a problem? Please describe:**
Users want to run model inference inside TiDB for data-locality, governance, and simplified ops. Today this requires external services and custom glue, which complicates auditing, configuration, and performance tuning.

**Describe the feature you'd like:**
- MODEL DDL to register model metadata and artifacts.
- MODEL_PREDICT execution with ONNX runtime backend.
- MLflow sidecar backend integration and config.
- Governance sysvars (enable/disable, batch size, timeout, null handling, cache settings).
- LLM builtins (llm_complete, llm_embed_text) with provider integrations (Vertex, Bedrock).
- Observability metrics and targeted tests.

**Describe alternatives you've considered:**
- External inference service called from the application layer.
- UDF/plugin-based inference.
These alternatives add operational complexity and don't provide integrated governance or optimizer awareness.

**Teachability, Documentation, Adoption, Migration Strategy:**
- Document MODEL DDL and MODEL_PREDICT usage with examples.
- Document sysvars and config sections for MLflow and LLM providers.
- Keep defaults disabled via sysvars so adoption is opt-in.

**Related PRs:**
- #66594 (core model serving: ONNX/MLflow)
- codex/llm-builtins (LLM builtins/providers; PR to be opened)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.