haskell / haskell/alex

Strict `Text` wrapper unnecessarily decodes to `Char` for `text-2.*`

Open
#280 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Haskell
Stars
323
Forks
87
PR merge metrics
No merged PRs in 30d

Description

Looking at the code:

https://github.com/haskell/alex/blob/e65958c688ba04f8b14c888d7b55f16532510f51/data/AlexWrappers.hs#L88-L90

This can be replaced by:

```haskell
type AlexInput = (Int, -- current offset
Data.Text.Text) -- input string
```

https://github.com/haskell/alex/blob/e65958c688ba04f8b14c888d7b55f16532510f51/data/AlexWrappers.hs#L98-L105

This can be replaced by:

```haskell
alexGetByte :: AlexInput -> Maybe (Byte,AlexInput)
alexGetByte (cur, input) = case input of
Text arr off len
| cur < len = Just (unsafeIndex arr (off + curr) (cur + 1, input))
| otherwise = Nothing
```

The only thing "complicated" is `alexInputPrevChar`, basically you go back from `cur` one byte at a time until you have seen two character boundaries and then do a character decode at that position.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.