matrixorigin / matrixorigin/matrixone

[Feature Request]: Optimize statement_info exec_plan storage by implementing conditional Physical Plan output

Open
#23,386 0 comments 0 reactions 1 assignee Claimed by @XuPeng-SH View on GitHub
kind/feature priority/p1
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

## Is there an existing issue for the same feature request?

- [x] I have checked the existing issues.

## Is your feature request related to a problem?

Currently, the `statement_info.exec_plan` field stores complete Physical Plan output for all slow queries, resulting in excessive storage usage. For typical TPCC workloads, each slow query can generate thousands of lines of Physical Plan details (5KB+ per query), causing significant storage bloat in the statement_info table.

Most queries don't need full Physical Plan details for basic performance analysis - the Overview section (CPU usage, memory, I/O statistics) provides sufficient information for most troubleshooting scenarios.

## Describe the feature you'd like

Implement a tiered output strategy for `statement_info.exec_plan` based on execution time thresholds:

## Describe implementation you've considered

## Documentation, Adoption, Use Case, Migration Strategy

## Additional information

**Benefits:**
- 80-90% reduction in statement_info storage for typical workloads
- Preserves all critical information for genuine slow queries
- Maintains backward compatibility
- No impact on `EXPLAIN PHY` command functionality
- No performance impact on query execution

**Testing Completed:**
- Verified compilation and existing test compatibility
- Confirmed EXPLAIN PHY commands unaffected
- Tested threshold logic with various execution times
- Validated Overview-only output contains sufficient performance data

**Priority**: Medium - Storage optimization with significant impact on operational costs

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.