opensearch-project / opensearch-project/sql

[FEATURE] JSON Search Support

Open
#2,652 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement SQL
Dominant language
Java
Stars
176
Forks
229
Avg merge
2d 21h
Merged PRs (30d)
43

Description

Is your feature request related to a problem?
Community members have asked for easier JSON parsing and analysis capabilities which allow them to not only search JSON logs and extract fields without writing complex parse expressions, but perform computations on JSON array values, such as finding the sum of all values in the array, where the number of elements in the array is not known.

What solution would you like?
Allow users to extract and transform data from JSON-formatted events and fields. Users should be able to extract all values in an array by specifying a wildcard for the individual element position and doing an aggregation operation on them. Users should be able to extract: 1/single or multiple top level fields 2/nested fields 3/keys in arrays and perform operations on the values.

** Examples ***

  • User wants to look at an array of elements, across log entries to see how many times each element has occurred in the array.
  • Match any item in a JSON array, regardless of the position in the array
  • Pull out fields within a nested JSON file

What alternatives have you considered?
No other solutions are available in PPL

Do you have any additional context?
No

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files, tests, or entry points. Start by locating the existing PPL JSON parsing and search implementation, then define support for top-level and nested fields, array wildcards, and aggregations; done requires these use cases to work across JSON log entries.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.