How does `tokenize` separate `SourceLine`s?
Open
- Dominant language
- Haskell
- Stars
- 218
- Forks
- 67
- PR merge metrics
- No merged PRs in 30d
Description
The function `tokenize` outputs an array of `SourceLine`s. Is it guaranteed that if i provide it with `Text` with some multiline code, every line's tokens will be mapped to one `SourceLine` without any formatting/line changing behaviour?
For example, if I had this single line of oddly written piece of C code:
```c
int do_something() { call_a_function(); int some_number = 3; return 123; }
```
Would very token (e.g. `int`, `do_something`, `()`, `{`, `call_a_function` ...) will end up on the same `SourceLine` with every token in sequence?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.