opensearch-project / opensearch-project/sql
Audit field availability for new keywords added with the 3.3 improvements
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 176
- Forks
- 229
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 43
Description
The Situation
This issue came up with #4185, #4113, #4120, and probably others:
In the PPL and SQL grammars, there's a keywordsCanBeId entry:
We've been adding a lot of new keywords to the Lexer, but haven't been adding them all to this entry. This means that you can't use those keywords as fields anymore without escaping. You get an error similar to this:
[field] is not a valid term at this part of the query: '...output=output inner.field' <-- HERE. Expecting tokens: ID
In the above case, the field keyword was added in #4109 and caused test failures for #4185 after merge, because field was used as a generic field name.
The Fix
We should audit the new keywords added as part of the 3.3 rush, and verify that all appropriate new keywords are also available as field names.
We could also consider finding a way to refactor such that we don't need to manually update this allowlist to begin with.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the keywordsCanBeId entry in ppl/src/main/antlr/OpenSearchPPLParser.g4 and compare it with the new keywords added to the PPL and SQL lexer grammars during the 3.3 improvements. Review the failures described in #4185 and related issues, then verify that appropriate keywords remain usable as field names without escaping. Done means the audit is complete and the affected parsing tests pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql
- Domain
- compilers, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100