matrixorigin / matrixorigin/matrixone
[Tech Request]: AI-Ready Observability
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
## What would you like to be added?
### Goal
Make MatrixOne observability data (metrics, logs, traces) AI-consumable, enabling AI Agents to autonomously diagnose cluster health, locate performance bottlenecks, and assist operational decisions.
This is a **Main Theme Issue**. Sub-tasks include but are not limited to:
### Sub-tasks
1. **Grafana Deployment & API Access**
- Expose Grafana HTTP / Data Source APIs for AI Agents to query panel data directly
- Review existing dashboards to ensure key metrics have structured, programmatic access paths
- Provide JSON/CSV metrics snapshot export capability
2. **Metrics Instrumentation Cleanup**
- Audit existing metrics; unify naming conventions (namespace, subsystem, name)
- Add missing instrumentation on critical paths (e.g. TN flush/merge latency, CN query queue time)
- Add description / help text to every metric so AI can understand semantics
3. **Structured Logs & Events**
- Emit structured logs for key operational events (compaction, GC, checkpoint, DDL changes, etc.)
- Provide an event stream interface or queryable system tables for AI time-series correlation
4. **AI-Friendly Metadata**
- Provide a metrics catalog / schema file describing each metric's meaning, unit, and normal range
- Provide structured cluster topology info (CN/TN/Log Service node list, roles, status)
5. **Query Interface**
- Provide a unified observability query API (or leverage MO's own SQL over internal system tables)
- Support AI Agents fetching diagnostic context via SQL or REST API
### Expected Outcome
- AI Agents can autonomously handle 80%+ routine cluster health checks and anomaly diagnosis
- Reduce manual Dashboard browsing for operators
- Lay the foundation for AI-driven auto-tuning and self-healing
## Why is this needed?
Current MO observability is designed for humans browsing Grafana dashboards, lacking programmatic / AI consumption design. As AI Agents land in operations scenarios, observability data needs to be:
- **Programmatically accessible**: via API/SQL, not UI-dependent
- **Semantically clear**: well-named metrics with descriptions and context
- **Structured**: parseable logs and events, not plain text
## Additional information
This is a long-term evolving theme. Each sub-task can be split into independent child issues and delivered incrementally.
Contributor guide
Assessment
This issue has not been assessed yet.