Regex inconsistencies
- Dominant language
- Haskell
- Stars
- 76
- Forks
- 11
- Avg merge
- 9h 5m
- Merged PRs (30d)
- 1
Description
Hello,
I've observed several inconsistencies between the regex pandoc uses when reading Typst documents and the regex Typst uses.
Here are a few of them:
1. Not all flags are supported. Typst regex supports the flags `i`, `m`, `s`, `u`, `x`. Of those, only `i` appears to be supported by Pandoc.
For example, `#(regex("(?m)a") in "A")` compiles in Typst, but doesn't in Pandoc (3.1.11.1 via try.pandoc.org), with the error `(line 1, column 2): parseRegex for Text.Regex.TDFA.Text failed:"({0,1}m)a" (line 1, column 4): unexpected '0' expecting an atom`.
- I especially miss the `m` (multiline) flag in order to be able to match the start of a line with `^` and the end of a line with `$`.
2. Unnamed capture groups are not supported: `#(regex("(?:x)") in "x")` compiles in Typst, but not in pandoc (`(line 1, column 2): parseRegex for Text.Regex.TDFA.Text failed:"({0,1}:x)" (line 1, column 4): unexpected '0' expecting an atom`).
- This is needed to avoid unnecessary capture groups in the output, and is frequently used across my packages.
3. Explicitly named capture groups are not supported: `#(regex("(?Px)") in "x")` compiles in Typst, but not in Pandoc (`(line 1, column 2): parseRegex for Text.Regex.TDFA.Text failed:"({0,1}Px)" (line 1, column 4): unexpected '0' expecting an atom`).
Besides non-compilation, there are inconsistencies in the results of regex matching as well.
1. `#(regex("[\s\S]+") in "x")` returns `true` in Typst, but `false` in Pandoc.
2. `#("a \n b" == "a \n b".match(regex("[^.]+")).text)` returns `true` in Typst, but `false` in Pandoc. In general, `[ ]` seems to unable to accept newlines, when it should.
There are probably inconsistencies I haven't found yet as well, but they could be added to this issue as they are found.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the Haskell regex handling used by the `regex` function and compare it with the listed Typst examples. Check flag support, capture-group syntax, and newline matching; done means the documented cases compile and produce matching results in Pandoc.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- haskell
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100