Jomy10 / Jomy10/nootlang

Fix tokenization for method call on integer

Open
#4 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Go
Stars
4
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Given the following code snippet:

```
a := 6.add(5)
```

The tokenizer will generate the following tokens:

`Ident(a), Declare(:=), Float(6.), Ident(add), OpenBracket((), Integer(5), CloseBracket())`

However, the `Float(6.)` token should be `Integer(6), Dot(.)`.

This requires either a rewrite of the tokenizer, disallowing `\d+\.` as a float, or tokenizing floats differently.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the tokenizer entry point and the logic that recognizes numeric literals. Reproduce tokenization for `a := 6.add(5)` and compare it with the expected Integer(6), Dot(.), Ident(add) sequence. Done means the integer method-call form no longer produces Float(6.) while ordinary float literals still tokenize correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
compilers
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.