elastic / elastic/roadmap

Serverless users can search across multiple projects

Open
#65 0 comments 0 reactions 1 assignee Assigned to @naj-h View on GitHub
Component: Elastic Cloud Serverless product-area:platform
Dominant language
No language data
Stars
6
Forks
1
PR merge metrics
No merged PRs in 30d

Description

**Value proposition**

Cross-Project Search (CPS) is a foundational feature reimagined for the Elastic Serverless platform, enabling Enterprise customers to seamlessly unify data scattered across multiple projects. This capability is critical for architectural requirements such as data residency, scalability, cost optimization, and tenant isolation.

The core value propositions delivered by CPS are:
1. Unified, "Flat-World" Search Experience: Searches initiated from an origin project automatically span all configured linked projects by default. This eliminates the need for users to modify every search request or manually manage project identifiers in queries.
2. Simplified Configuration: Configuration is streamlined to a single-step process in the Cloud UI/API, avoiding the complex, error-prone certificate, IP address, and API key management required by traditional Cross-Cluster Search (CCS).
3. Consistent and Secure Data Access: Utilizing the new Universal Identity and Access Management (UIAM) model, CPS ensures that access permissions to data are consistent, regardless of whether the user accesses the data directly from the linked project or via the origin project (the core tenant of the new security model).
4. Cross-Solution Analysis: CPS supports linking Observability (O11y) and Security projects, enabling use cases such as analyzing O11y logs to find security threats, capturing the value of cross-solution capabilities.

**Expected outcome**

The initial phase of Cross-Project Search (CPS) will deliver foundational capabilities that redefine unified data access across multiple Elastic Serverless projects:
1. Unified, "Flat-World" Search Experience:
◦ The system will introduce a "flat-world" mental model where remote data is treated identically to local data for search operations.
◦ Searches initiated from an origin project will span all configured linked projects by default, eliminating the need for users to prefix queries with project names.
◦ Unqualified index expressions (e.g., FROM foo) will implicitly refer to that resource on the origin project and all linked projects that contain a matching resource.
◦ The architecture will actively prevent cross-project chaining in this initial phase; searches will only span the origin project and its directly linked projects.
2. Simplified Configuration and Management:
◦ A streamlined, centralized interface (Cloud UI/API) will be delivered for linking projects, replacing the multi-step, error-prone configuration (certificates, API keys) required by traditional Cross-Cluster Search (CCS).
◦ Configuration information will be propagated securely to Elasticsearch clusters using established Kubernetes configuration patterns, allowing Elastic to hot-reload link configurations without service interruption.
◦ Administrators can define linked projects using project names or metadata attributes, but only a fixed list of project IDs will be saved initially.
◦ Configuration will be strictly unidirectional: configuring Project A to search Project B does not automatically enable Project B to search Project A.
3. Consistent and Secure Access Control (UIAM):
◦ The implementation will leverage the new Universal Identity and Access Management (UIAM) security model.
◦ Access permissions to data will be enforced consistently, regardless of whether the user accesses the data directly from the linked project or via the origin project .
◦ Support for Universal API keys will enable client applications and critical Kibana background tasks (like alerting rules) to search across configured projects programmatically using a single, multi-project credential.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.