matrixorigin / matrixorigin/matrixone

[Tech Request]: AI-Ready Observability

Open
#24,222 0 comments 0 reactions 1 assignee Claimed by @aptend View on GitHub
kind/tech-request
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

## What would you like to be added?

### Goal

Make MatrixOne observability data (metrics, logs, traces) AI-consumable, enabling AI Agents to autonomously diagnose cluster health, locate performance bottlenecks, and assist operational decisions.

This is a **Main Theme Issue**. Sub-tasks include but are not limited to:

### Sub-tasks

1. **Grafana Deployment & API Access**
- Expose Grafana HTTP / Data Source APIs for AI Agents to query panel data directly
- Review existing dashboards to ensure key metrics have structured, programmatic access paths
- Provide JSON/CSV metrics snapshot export capability

2. **Metrics Instrumentation Cleanup**
- Audit existing metrics; unify naming conventions (namespace, subsystem, name)
- Add missing instrumentation on critical paths (e.g. TN flush/merge latency, CN query queue time)
- Add description / help text to every metric so AI can understand semantics

3. **Structured Logs & Events**
- Emit structured logs for key operational events (compaction, GC, checkpoint, DDL changes, etc.)
- Provide an event stream interface or queryable system tables for AI time-series correlation

4. **AI-Friendly Metadata**
- Provide a metrics catalog / schema file describing each metric's meaning, unit, and normal range
- Provide structured cluster topology info (CN/TN/Log Service node list, roles, status)

5. **Query Interface**
- Provide a unified observability query API (or leverage MO's own SQL over internal system tables)
- Support AI Agents fetching diagnostic context via SQL or REST API

### Expected Outcome

- AI Agents can autonomously handle 80%+ routine cluster health checks and anomaly diagnosis
- Reduce manual Dashboard browsing for operators
- Lay the foundation for AI-driven auto-tuning and self-healing

## Why is this needed?

Current MO observability is designed for humans browsing Grafana dashboards, lacking programmatic / AI consumption design. As AI Agents land in operations scenarios, observability data needs to be:
- **Programmatically accessible**: via API/SQL, not UI-dependent
- **Semantically clear**: well-named metrics with descriptions and context
- **Structured**: parseable logs and events, not plain text

## Additional information

This is a long-term evolving theme. Each sub-task can be split into independent child issues and delivered incrementally.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.