Fix tokenization for method call on integer
- Dominant language
- Go
- Stars
- 4
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Given the following code snippet:
```
a := 6.add(5)
```
The tokenizer will generate the following tokens:
`Ident(a), Declare(:=), Float(6.), Ident(add), OpenBracket((), Integer(5), CloseBracket())`
However, the `Float(6.)` token should be `Integer(6), Dot(.)`.
This requires either a rewrite of the tokenizer, disallowing `\d+\.` as a float, or tokenizing floats differently.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the tokenizer entry point and the logic that recognizes numeric literals. Reproduce tokenization for `a := 6.add(5)` and compare it with the expected Integer(6), Dot(.), Ident(add) sequence. Done means the integer method-call form no longer produces Float(6.) while ordinary float literals still tokenize correctly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100