opensearch-project / opensearch-project/sql
[FEATURE] PPL: Gracefully handle non-existent fields in queries instead of throwing errors
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 176
- Forks
- 229
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 43
Description
Is your feature request related to a problem?
PPL queries fail when attempting to reference non-existent fields, particularly problematic when working with OpenTelemetry (OTEL) data where not all standard fields may be instrumented.
What solution would you like?
Modify the PPL query engine to gracefully handle non-existent fields by ignoring them instead of throwing errors. This would allow queries to:
- Execute successfully even when referencing fields that don't exist in the index
- Return results based on the fields that do exist
- Skip or ignore clauses referencing non-existent fields
What alternatives have you considered?
Current workaround requires additional preprocessing steps:
- Fetching available fields using index pattern API:
const indexPattern = await data.indexPatterns.get(indexPatternId);
const fieldNames = indexPattern.fields;
- Filtering query clauses to only include existing fields This creates unnecessary overhead in query generation and makes the code more complex.
Do you have any additional context?
- This is particularly important for OpenTelemetry use cases where the standard defines many possible fields, but applications may only instrument a subset
- Similar functionality exists in other query languages (e.g., Elasticsearch's exists query)
- Would improve user experience by removing the need to know exact schema before querying
- Current behavior forces users to write defensive code and add complexity to their applications
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the PPL query engine entry point and tracing how references to non-existent fields currently produce errors. Define how missing fields should be handled across filters and other clauses, then verify that queries return results from existing fields without throwing. No specific files or tests are named in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100