eclipse-jdt / eclipse-jdt/eclipse.jdt.core
InvalidInputException when using Scanner on module import declaration
- Dominant language
- Java
- Stars
- 237
- Forks
- 195
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 47
Description
When the Java token scanner gets to an `import module` token, it throws a `InvalidInputException: Unknown token (check Scanner/TerminalToken): TokenNamemodule`.
For example, consider the following test:
```java
@Test
public void testInvalidTokenName() throws InvalidInputException {
String s = """
import module java.base;
int i;
int j;
void main() {
// main
}
""";
IScanner scanner= ToolFactory.createScanner(true, false, false, false);
scanner.setSource(s.toCharArray());
do {
// nothing
} while (scanner.getNextToken() != ITerminalSymbols.TokenNameEOF);
}
```
This test fails with an exception like this:
```
org.eclipse.jdt.core.compiler.InvalidInputException: Unknown token (check Scanner/TerminalToken): TokenNamemodule
at org.eclipse.jdt.internal.compiler.parser.Scanner.invalidToken(Scanner.java:5565)
at org.eclipse.jdt.internal.core.util.PublicScanner.translateTokenToTerminalSymbol(PublicScanner.java:271)
at org.eclipse.jdt.internal.core.util.PublicScanner.getNextToken(PublicScanner.java:118)
at org.eclipse.jdt.text.tests.folding.FoldingTest.testInvalidTokenName(FoldingTest.java:861)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
```
I think the `ITerminalSymbols` class needs a `TokenNameModule` constant and `PublicScanner#translateTokenToTerminalSymbol` should be updated to use that. I guess that method could be refactored to make use of an exhaustive switch so this doesn't happen in the future?
Contributor guide
Assessment
This issue has not been assessed yet.