JetBrains / JetBrains/Grammar-Kit
Should we have comments in BNF?
- Dominant language
- Java
- Stars
- 771
- Forks
- 137
- Avg merge
- 11h 14m
- Merged PRs (30d)
- 1
Description
PsiBuilder ignores whitespace and comment tokens but we have to use comment token in BNF just to have corresponding IElementType generated.
Even the official documentation for [Custom Language](https://plugins.jetbrains.com/docs/intellij/grammar-and-parser.html#define-the-grammar) has it as example: `private item_ ::= (property|COMMENT|CRLF)`.
In my own grammar I've ended up with an 'orphan' rule and it works fine: `private comment ::= COMMENT`
If comment tokens doesn't add anything to a parser, shouldn't we stop using them as we do for whitespace tokens. Of course they are necessary for corresponding IElementType generation but we could use an 'orphan' rules for them or even have a predefined `com.intellij.psi.TokenType`.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the Grammar-Kit handling of BNF comment tokens and the IntelliJ grammar-and-parser documentation linked in the issue. Compare the documented `COMMENT` usage with the proposed orphan rule and predefined `TokenType` alternatives; done means an agreed direction for comment-token handling is documented or implemented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- compilers
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100