denoland / denoland/std

Low-level Unicode code-point-based functions

Open
#6,627 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
3.6k
Forks
681
PR merge metrics
No merged PRs in 30d

Description

**Is your feature request related to a problem? Please describe.**

For reasons of interoperability etc. it's often useful to know how long a string is, compare string length, collate strings, or index into a string based on its code points.

The built-in JS functionality for this is instead based on UTF-16 code units, which give the same results for many but not all strings.

Meanwhile, it's trivial to split a string into its constituent code points using `Array.from` or similar, but doing so generally isn't very efficient compared to more optimized alternatives.

**Describe the solution you'd like**

Low-level, well optimized, but fully Unicode code point aware functions such as:
* String length
* String length comparison[^1]
* Indexing by code point[^2]
* Two-way conversion between code-point index and code-unit index[^2]
* String comparison by code point ([stage-1 proposal](https://github.com/tc39/proposal-compare-strings-by-codepoint))
* Maybe others?

**Describe alternatives you've considered**

N/A

[^1]: String length comparison vs a limit can often be calculated more efficiently than fully calculating the actual string length and comparing it to the limit. For example, for `limit=1000`, `text.length == 2001` definitely exceeds it, while `text.length == 999` is definitely lower (whereas `text.length` in the range `1000..2000` needs to be checked more carefully).
[^2]: Not sure how useful these are actually. Indexing is usually best done in code-units for compatibility with other JS in-built functions. However, there could still be interoperability use cases for them?

Contributor guide

Open the contributing guide

Research direction

The issue names no repository files, tests, or entry points. Start by reviewing the existing JavaScript string behavior and the Array.from alternative, then clarify which code-point operations belong in the library. Done requires an agreed API and scope for the requested Unicode-aware functions.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, typescript
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.