julia-script / julia-script/silk
stdlib: add grapheme traversal and locale-independent case mapping
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 48
- Forks
- 0
- Avg merge
- 4h 49m
- Merged PRs (30d)
- 213
Description
Context
silk/unicode now ships Unicode 17.0.0 NFC and NFD normalization. This issue tracks the
remaining post-stable Unicode functionality: extended grapheme-cluster traversal and full
locale-independent default upper/lower case mapping.
Locale-sensitive collation remains tracked separately in #162.
Current behavior
import silk.unicode { normalizeNfc, normalizeNfd }
Normalization is available and ordinary string equality remains exact. There is currently no
grapheme cursor and no upper- or lower-case transformation API.
Intended surface
pub fn nextGrapheme(
value: string,
cursor: GraphemeCursor
) -> Option<GraphemeStep>
pub effect fn toLowercase(
value: string
) -> String ! OutOfMemoryError ? &mut Allocator
pub effect fn toUppercase(
value: string
) -> String ! OutOfMemoryError ? &mut Allocator
GraphemeStep reports byte start/end offsets and the next cursor. It does not retain a borrowed
string view.
Requirements
- Grapheme traversal implements Unicode 17.0.0 extended grapheme clusters from UAX #29.
- Traversal is allocation-free and reports UTF-8 byte boundaries.
- Case conversion implements full locale-independent default lower/upper mappings, including
contextual mappings. - Case conversion allocates an owned
Stringand exposesOutOfMemoryErrorplusAllocator. - Language tailoring, case folding, title casing, and locale-sensitive collation are out of scope.
- Tables remain generated ordinary Silk data with no compiler privilege.
- Ordinary
stringequality remains exact and normalization-free.
Acceptance criteria
- A base scalar and combining mark remain one grapheme.
- Emoji ZWJ and regional-indicator conformance cases match Unicode 17.0.0 data.
- Traversal returns exact UTF-8 byte offsets without allocating.
- Lower/upper conversion covers a mapping that changes byte length.
- Contextual default mappings are covered.
- Evaluator, native LLVM, and direct Wasm agree.
- Generation is deterministic and the compiler contains no Unicode-policy special case.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the silk/unicode implementation and the existing normalizeNfc and normalizeNfd surfaces. Review the proposed nextGrapheme, toLowercase, and toUppercase behavior against Unicode 17.0.0 and UAX #29, then verify agreement across the evaluator, native LLVM, and direct Wasm. Done means the listed grapheme, mapping, allocation, generation, and exact-equality criteria pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- compilers
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100