jgm / jgm/typst-hs

Regex inconsistencies

Open
#28 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Haskell
Stars
76
Forks
11
Avg merge
9h 5m
Merged PRs (30d)
1

Description

Hello,
I've observed several inconsistencies between the regex pandoc uses when reading Typst documents and the regex Typst uses.

Here are a few of them:

1. Not all flags are supported. Typst regex supports the flags `i`, `m`, `s`, `u`, `x`. Of those, only `i` appears to be supported by Pandoc.
For example, `#(regex("(?m)a") in "A")` compiles in Typst, but doesn't in Pandoc (3.1.11.1 via try.pandoc.org), with the error `(line 1, column 2): parseRegex for Text.Regex.TDFA.Text failed:"({0,1}m)a" (line 1, column 4): unexpected '0' expecting an atom`.
- I especially miss the `m` (multiline) flag in order to be able to match the start of a line with `^` and the end of a line with `$`.
2. Unnamed capture groups are not supported: `#(regex("(?:x)") in "x")` compiles in Typst, but not in pandoc (`(line 1, column 2): parseRegex for Text.Regex.TDFA.Text failed:"({0,1}:x)" (line 1, column 4): unexpected '0' expecting an atom`).
- This is needed to avoid unnecessary capture groups in the output, and is frequently used across my packages.
3. Explicitly named capture groups are not supported: `#(regex("(?Px)") in "x")` compiles in Typst, but not in Pandoc (`(line 1, column 2): parseRegex for Text.Regex.TDFA.Text failed:"({0,1}Px)" (line 1, column 4): unexpected '0' expecting an atom`).

Besides non-compilation, there are inconsistencies in the results of regex matching as well.
1. `#(regex("[\s\S]+") in "x")` returns `true` in Typst, but `false` in Pandoc.
2. `#("a \n b" == "a \n b".match(regex("[^.]+")).text)` returns `true` in Typst, but `false` in Pandoc. In general, `[ ]` seems to unable to accept newlines, when it should.

There are probably inconsistencies I haven't found yet as well, but they could be added to this issue as they are found.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the Haskell regex handling used by the `regex` function and compare it with the listed Typst examples. Check flag support, capture-group syntax, and newline matching; done means the documented cases compile and produce matching results in Pandoc.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.