haskell / haskell/text

Optimize encodeUtf8 by checking if the byte array is already pinned

Open
#471 2 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
Haskell
Stars
421
Forks
163
PR merge metrics
No merged PRs in 30d

Description

The byte array underlying `Text` [might already be pinned](https://ghc.gitlab.haskell.org/ghc/doc/users_guide/exts/ffi.html#pinned-byte-arrays):

> A byte array [and by extension Text] can be pinned as a result of three possible causes:
>
> 1. It was allocated by newPinnedByteArray#. [Text is never constructed using this primitive]
> 2. It is large. Currently, GHC defines large object to be one that is at least as large as 80% of a 4KB block (i.e. at least 3277 bytes).
> 3. It has been copied into a compact region. The documentation for ghc-compact and compact describes this process.

Especially 2 seems likely and a case where we could save a significant amount of time by avoiding copying. Could we use [`isByteArrayPinned#`](https://hackage.haskell.org/package/base-4.14.1.0/docs/GHC-Exts.html#v:isByteArrayPinned-35-) to check if the byte array is already pinned and in that case avoid copying?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at encodeUtf8 and the Text byte-array representation, then read GHC's pinned-byte-array documentation and the isByteArrayPinned# reference. Done means avoiding the copy when the underlying byte array is already pinned while preserving UTF-8 encoding behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell
Domain
performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.