JuliaLang / JuliaLang/JuliaSyntax.jl

AST node for iteration

Open
#432 1 comment 1 reaction 0 assignees View on GitHub
AST breaking
Dominant language
Julia
Stars
293
Forks
50
PR merge metrics
No merged PRs in 30d

Description

There's been something niggling at me about the way iteration is represented in the AST.

Currently we have the following parsing:

```julia
julia> parsestmt(SyntaxNode, """
for x = xs
body
end""")
line:col│ tree │ file_name
1:1 │[for] │
1:4 │ [=]
1:5 │ x
1:9 │ xs
1:11 │ [block]
2:4 │ body

julia> parsestmt(SyntaxNode, """
for x = xs, y = ys
body
end""")
line:col│ tree │ file_name
1:1 │[for] │
1:4 │ [cartesian_iterator]
1:4 │ [=]
1:5 │ x
1:9 │ xs
1:12 │ [=]
1:13 │ y
1:17 │ ys
1:19 │ [block]
2:4 │ body
```

But the `=` node here doesn't have normal assignment semantics. It *does* create a binding for `x`, but not to the expression on the right hand side of the `=`. Also the user may use `in` rather than `=` in the source; this is normalized to `=` by the parser for consistency, but this only emphasizes that there's something a bit weird going on: it's not assignment; merely assignment-like.

The use of `cartesian_iterator` is semantically nice because we also get to reuse it for array comprehensions where it means the same thing (the representation in `Expr` doesn't have this level of uniformity so things are already, hopefully, a bit of an improvement).

Complex comprehensions only make this worse, where `=` nodes can appear all over the place in the AST. For example,

```
julia> parsestmt(SyntaxNode, """
[a for i = xs, j = ys if z]""")
line:col│ tree │ file_name
1:1 │[comprehension] │
1:2 │ [generator]
1:2 │ a
1:7 │ [filter]
1:7 │ [cartesian_iterator]
1:7 │ [=]
1:8 │ i
1:12 │ xs
1:15 │ [=]
1:16 │ j
1:20 │ ys
1:26 │ z

julia> parsestmt(SyntaxNode, """
[a for i = xs for j = ys if z]""")
line:col│ tree │ file_name
1:1 │[comprehension] │
1:2 │ [generator]
1:2 │ a
1:7 │ [=]
1:8 │ i
1:12 │ xs
1:18 │ [filter]
1:18 │ [=]
1:19 │ j
1:23 │ ys
1:29 │ z
```

### Possible solution

I'd like to propose a syntax kind `K"iteration"` to replace the use of both `K"="` and `K"cartesian_iterator"`.

A possible rule could be:
* Existing `(= x xs)` AST for iteration becomes `(iteration x xs)`
* Existing `(cartesian_iterator (= x xs) (= y ys))` becomes `(iteration x xs y ys)`.

The cases above would look like

```
julia> parsestmt(SyntaxNode, """
[a for i = xs, j = ys if z]""")
line:col│ tree │ file_name
1:1 │[comprehension] │
1:2 │ [generator]
1:2 │ a
1:7 │ [filter]
1:7 │ [iteration]
1:8 │ i
1:12 │ xs
1:16 │ j
1:20 │ ys
1:26 │ z

julia> parsestmt(SyntaxNode, """
[a for i = xs for j = ys if z]""")
line:col│ tree │ file_name
1:1 │[comprehension] │
1:2 │ [generator]
1:2 │ a
1:7 │ [iteration]
1:8 │ i
1:12 │ xs
1:18 │ [filter]
1:18 │ [iteration]
1:19 │ j
1:23 │ ys
1:29 │ z
```

Some possible advantages
* a semantically sound syntax kind fixes the awkwardness of preferring `=` vs `in` for a concept which is neither really assignment nor set inclusion
* This makes recursive processing of the syntax tree a bit simpler if one wants to find `=` but omit special-casing the first child of `for`
* Possibly makes writing macros simpler when processing cartesian iterators, as there's less nesting? Unclear.
* Renaming `cartesian_iterator` to something less specific feels sane because cartesian iteration is a very semantically precise/narrow concept, but AST for surface syntax is, somewhat, just trying to represent what's there visually. For macros to process, for example.

#### Alternatives?

Is allowing `iteration` to have any even number of children to represent cartesian iteration a good call? Would writing macro code against a nested AST be easier?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.