coderkalyan / coderkalyan/femtoc

lex: consolidate literal parsing into lexer

Open
#51 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Zig
Stars
2
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Currently, literal parsing is handled in two stages. Initial discrimination is handled by the lexer, but full sanitization and parsing (calculating the underlying value) is handled by the AST. This is wasteful, and buggy because not all invalid literal tokens are rejected by the lexer.

A better solution is to integrate the integer and float literal state machines into the lexer, and have it append the calculated literal to a ArrayList(u64) literal "tape". The parser can then pop literals off this when it encounters a corresponding .int_lit or .float_lit token.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.