[Feature request] Additional parsing functions for the 'yue'-module
Nobody has claimed this yet.
- Dominant language
- Lua
- Stars
- 613
- Forks
- 45
- PR merge metrics
- No merged PRs in 30d
Description
It would be nice to have the following functions be added to the yue module:
yue.to_tokens(code: string): string[]: Turns a chunk of YueScript code into a list of tokens, solocal even_nums = [i for 1, 10 when (i % 2) == 0]would become something like[ "local", " ", "even_nums", " ", "=", " ", "[", "i", " ", "for", " ", "i", " ", "=", " ", "1", ",", " ", "10", " ", "when", " ", "(", "i", " ", "%", " ", "2", ")", " ", "==", " ", "0", "]" ]. This would be particularly useful for macros because the input might not make sense when parsed into a YueScript AST.yue.to_cst(code: string): yue.CstNode: A CST (concrete syntax tree) is similar to an AST, but unlike an AST, it contains all information required to reconstruct the exact source code that it was generated from. An AST would discard information like whitespace and comments, while a CST would keep them. A CST should also ideally be a strict superset of an AST, so that a node visitor for an AST can also operate on the same node when used with a CST.
As a side note: It may be worth considering replacing the current macro semantics with something more like Rust's approach of taking a single list of tokens and returning a list of tokens, instead of working with source code. Doing it this way is easier to use as well as more reliable than trying to monkey-patch some strings with Lua patterns / regular expressions.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the yue module, its current parser, and the macro entry points mentioned in the issue. Define how token lists and a CST should represent the supplied YueScript example, including whitespace and comments. Done means the module exposes both parsing functions and the CST can preserve enough information to reconstruct the original source.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- lua
- Domain
- compilers
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100