JuliaLang / JuliaLang/JuliaSyntax.jl

Macro expansion and lowering

Open
#327 6 comments 8 reactions 0 assignees View on GitHub
design
Dominant language
Julia
Stars
293
Forks
50
PR merge metrics
No merged PRs in 30d

Description

As requested in https://github.com/JuliaLang/JuliaSyntax.jl/issues/312#issuecomment-1616377851, here's a concrete description of roughly what needs doing (in reference to files over at JuliaLang/julia) to tackle the next parts of the compiler frontend.

* Prototype rewriting macro expansion in src/ast.c and src/macroexpand.scm. (However not the parts which deal with resolving `GlobalRef`s - those are badly placed in the compilation pipeline and should not be copied). I'm a good way through this on the caf/macro-expansion branch.
* Try out some experiments with new macro expansion. It should
- Preserve source location information provided by the parser
- Allow user-written macros to report errors in expressions using that precise source location information. Kinda like this https://twitter.com/c42f_/status/1678521992507543553
- Solve https://github.com/JuliaLang/julia/issues/37691
- Finally implement a version of "automatic hygiene" as in https://github.com/JuliaLang/julia/pull/6910, if possible.
- Have some opt-in based migration path which doesn't depend on getting to Julia 2.0
* Prototype rewriting "some representative parts" of lowering from src/julia-syntax.scm. Investigate what it takes to preserve source location information in error messages.
* Use these experiments to figure out what kind of syntax tree data structures and APIs we need.
- For data structures, will small modifications to `SyntaxNode` do the trick? Or do we need something else? What about storing optional semantic annotations using an ECS data structure like in Overseer.jl?
- For APIs, are we happy with manually branching on `head(expression)` and relying on the ordering of AST children? Do we want pattern matching? Do we want accessors? See #230
- How do we use these data structures to "talk to the user about their code"? Even after lowering has sliced and diced and rearranged the expression tree and maybe lowered it to a linearized `CodeInfo`-like form?
* Figure out how to proceed compatibly, supporting both `Expr` and `SyntaxNode` (or similar) through lowering. "A big switch over in Julia 2.0" will never work here. The only way to succeed in this project is to introduce the new stuff gradually in a compatible opt-in way. And maybe eventually have it on by default in Julia 2.0.
* Make some changes in Base to add hooks to replace all or parts of lowering, so a new macro expansion / lowering can be experimented with easily without need to monkey patch Base.

Having tried things and worked out some rough answers to those questions, the long grind of rewriting and testing all of lowering can go ahead.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.