elastic / elastic/roadmap

Stateless for Self-Managed: Easier to operate, reduced indexing costs

Open
#104 0 comments 0 reactions 1 assignee Claimed by @yuvielastic View on GitHub
Component: ECK Component: Elasticsearch product-area:platform
Dominant language
No language data
Stars
6
Forks
1
PR merge metrics
No merged PRs in 30d

Description

**Description:**

This outcome solves the critical problem of high TCO and inflexible scaling for large, self-managed deployments. Building upon segment-based replication, it allows Self-Managed customers to use low-cost object storage (e.g., S3, GCS, Azure Blob) as the system of record and scale indexing and search compute resources independently. This dramatically reduces indexing costs and provides the operational flexibility needed to manage petabyte-scale environments efficiently.

**Key Capabilities:**

- Enable configuration of certified S3-compatible object store as the primary repository for data, where users wish to use non-CSP object stores and keep data on-prem.
- Enable the use of the 3 major CSP Object Stores.
- Provide distinct node roles or tiers to allow for the independent scaling of indexing compute and search compute.
- Develop UI (in Kibana) and APIs for configuring and managing object store connections and compute tiers.
- Provide clear migration and upgrade paths for existing Self-Managed Enterprise customers to adopt the stateless architecture.
- Improved Sharding experience, with new APIs to split & shrink shards in place.

Stateless for Self-Managed will be packaged as an Enterprise feature and be distributed via the usual Elastic Stack packages. ECK will also be capable of running Stateless clusters.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.