antlr / antlr/antlr5

Design - Compile the runtime Lexer to WebAssembly

Open
#32 16 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
75
Forks
8
PR merge metrics
No merged PRs in 30d

Description

Beyond tooling issues, we also need to deal with paradigms that cannot work with WebAssembly.

In the current runtime, the Lexer is an abstract class, and the generated actual XXXLexer inherits from it.
This paradigm won't work with WebAssembly, especially not across language targets.

Looking at a generated XXXLexer, it doesn't provide behavior, rather it provides data that the runtime Lexer will use.
Therefore an idea that comes to mind is to evolve the design as follows:
- the generated XXXLexer becomes a standalone class that:
- provides data to a runtime Lexer instance
- forwards calls such as nextToken to that Lexer instance
- the data itself sits in a LexerData record (data class in Kotlin)
- the runtime Lexer becomes a concrete class that requires a LexerData record when instantiated

My plan is to first make the above work in Kotlin, then compile to Wasm.
Your comments on the proposed design are welcome.

Contributor guide

Open the contributing guide

Research direction

Start by locating the Kotlin runtime Lexer and a generated XXXLexer, then inspect how the generated class supplies data and forwards nextToken calls. Prototype the proposed LexerData and concrete runtime design in Kotlin, and verify that the resulting runtime Lexer and generated lexer compile to WebAssembly.

Written by the indexing model from the issue text.

Assessment

Tech stack
kotlin, wasm
Domain
compilers
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.