elastic / elastic/roadmap

ES|QL: Access unmapped data in flattened fields

Open
#108 0 comments 0 reactions 1 assignee Assigned to @yannis-roussos View on GitHub
Component: Elasticsearch product-area:observability product-area:search product-area:security v9.5.0
Dominant language
No language data
Stars
6
Forks
1
PR merge metrics
No merged PRs in 30d

Description

## Value Prop

Today it is very hard to access and analyze data that reside in unmapped fields as both `_search` and ES|QL require fields to be mapped in order to be used in queries. Having not fully mapped data more often happens with logs, which are often unstructured and composed of too many variations of fields. Users have to parse each field that they want to be able to search and analyze.

With this effort, we enable users with data that is stored in Elasticsearch but might not be fully mapped to be able to access all fields in a similar way so that they can search and analyze unmapped data.

## Expected Outcome

ES|QL will allow accessing and searching flattened fields
- Fields mapped as `flattened` can be loaded and, together with the new `FIELD_EXTRACT` ES|QL function, queried by sub-field
- Load the complete flattened object (JSON Blob) or extract a specific value: `FIELD_EXTRACT(resource.attributes, "host.name")`
- Sub-fields use their exact, case-sensitive dotted key.
- Extracted values are returned as keyword.
- When a sub-field holds multiple values, the result is a multi-valued keyword.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.