effekt-lang / effekt-lang/effekt

RFC: Raw multi-line strings delimited by a prefix token

Open
#1,128 0 comments 0 reactions 0 assignees View on GitHub
area:parser/lexer experiment requires-design
Dominant language
Scala
Stars
469
Forks
41
Avg merge
1d 16h
Merged PRs (30d)
23

Description

> [!NOTE]
> This is a request for comments in very early stages, feel free to comment/nitpick below.

Inspired by [Zig](https://ziglang.org/documentation/master/#Multiline-String-Literals) and [MoonBit](https://docs.moonbitlang.com/en/latest/language/fundamentals.html#string), use a prefix operator for denoting a multi-line string, which has a few advantages: easier to lex [untested?], each token spanning at most a single line [ => ability to lex each line separately ], and the allowing the user to have control over indentation of the prefix.

The main disadvantage is that this spends more of the syntax weirdness budget on something unnecessary, confusing researchers, students, and LLMs alike.

## A preliminary design

Multi-line strings are denoted with `#|`, are _raw_ (no escapes), but allow splices:
```scala
val greeting =
#|Hello ${name}!
#|Welcome to our application.
#| Today is ${do datetime().day.format} and the emoji of the day is #|--|#...\nCool, huh?

// ~> output:

//Hello Maxine!
//Welcome to our application.
// Today is Tuesday and the emoji of the day is #|--|#...\nCool, huh?
```

We can also support multi-line splices by adding a `$|` token that denotes currently being in a splice:
```scala
val template =
#|


#|

${user.name}


#|

Items:

    ${
    $| user.items.map { item =>
    $| "
  • ${item.title}
  • "
    $| }.join("\n")

$| }


#|

```

and we can even nest multi-line strings in multi-line splices, were we so inclined (number of `$|`'s marks the nesting depth of the "current" splice)
```scala
def messageFor(user: User): String =
#|Welcome ${user.name}!
#|
#|Your personalized content: ${
$| generateContent(
$| #| Dear ${user.name},
$| #| Here's your nested message: ${
$| $| processData(user.preferences)
$| $| }
$| #| Best regards!
$| )
$|}
#|Thanks for visiting!
```

A nice thing is that we can still support multi-line splices, but we can hope that our users won't need to learn `$|` and the nesting rules unless they really, really want to.

Together with this, I'd also be in favour of banning multi-line splices in single-line strings completely + revising hole string syntax to be [visually] compatible.

### An alternative design

We could also use a Scala-inspired design and reify the `|` there, reserving it as a prefix operator.
```scala
"""|Hello ${name}!
|Welcome to our application.
| Today is ${do datetime().day.format} and the emoji of the day is #|--|#...\nCool, huh?
|"""
```

### VSCode extension changes required

In addition to the syntax highlighting changes necessary, for ergonomics, I assume that the VSCode extension will do the following (`ⵋ` being the cursor):
```scala
#|Helloⵋ // <- press enter here

// ~>

#|Hello
#|ⵋ
```

```scala
#|Hello ${ⵋ // <- press enter here

// ~>

#|Hello ${
$|ⵋ
```

```scala
#|Hello ${name}ⵋ // <- press enter here

// ~>

#|Hello ${name}
#|ⵋ
```

and so on...

### String interpolation

It might be nice to have an additional `newline` effect here to let the handler choose what to do when there's a new line.
(For situations such as markdown where two newlines have a semantic meaning of their own.)

No idea about the concrete syntax: we could keep the current `foo#|...` syntax, but we could also try something else...

### Externs

This should be immediately usable for extern splices, and could potentially resolve some problems related to whitespace sensitivity [?]

## First steps

It would be nice to do an experiment: implement this in the lexer, independent of the existing string-related machinery (if possible), and see how this feels :)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.