coderkalyan / coderkalyan/femtoc

lex: support special characters and unicode

Open
#45 0 comments 0 reactions 0 assignees View on GitHub
good first issue
Dominant language
Zig
Stars
2
Forks
0
PR merge metrics
No merged PRs in 30d

Description

The lexer should support both ascii special (escaped) characters in strings and characters, and in general unicode support for strings and potentially identifiers. We need to investigate what other modern languages (zig, rust, hare) do for this - I know hare is quite unicode friendly. We aren't looking for general unicode "understanding" in femto with runes, etc right now, but the lexer shouldn't reject internationally written code.

This should also be fuzzed, pending #46

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.