elastic / elastic/roadmap

Materialized Views: Accelerate queries by defining automatically refreshing, pre-calculated views

Open
#49 0 comments 1 reaction 1 assignee Assigned to @tylerperk View on GitHub
Component: Elasticsearch Component: Kibana product-area:observability product-area:search product-area:security
Dominant language
No language data
Stars
6
Forks
1
PR merge metrics
No merged PRs in 30d

Description

**Value Proposition**

Queries can get complex and can take longer to run than a user loading a dashboard would like to wait. For popular queries used frequently in dashboards or alerting and detection rules, it pays to accelerate them by pre-calculating the results, like we do with Transforms. Views can store a query for easy reuse, and by automatically running the view on a recurring basis, Materialized Views make them faster.

**Expected outcome**

Address the critical problem of slow dashboards and analytical queries over large datasets. Pre-calculate and store the results of resource-intensive ES|QL queries, with a mechanism to automatically keep the stored data refreshed. This transforms query performance from minutes to seconds, improving the user experience for Security and Observability use cases.

**Key Capabilities:**

Implement CREATE MATERIALIZED VIEW mv_name ... AS ... syntax.
Include a REFRESH policy clause (e.g., REFRESH EVERY 5 MINUTES) to control data freshness.
Develop a background process to automatically execute refreshes according to the policy.
Store pre-calculated results in a managed Elasticsearch index.
Provide management commands like SHOW MATERIALIZED VIEWS, DROP MATERIALIZED VIEW, and ALTER MATERIALIZED VIEW (to change refresh policy).
Handle refresh failures and provide monitoring capabilities for refresh status and resource cost.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.