alecthomas / alecthomas/participle

Panic when stateful lexer's non-Root rule has optional group but captures nothing

Aperta
#324 4 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Go
Stelle
3.9k
Fork
213
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

![](https://user-images.githubusercontent.com/7822577/225385498-6d1455d4-06b9-4266-984d-e19e54ad354d.png)

Above example shows a common design: assume we have some keyword (`x` in this example), so it can not be parsed as a `Ident` in whole program, we want use a special symbol `%` to remove this limit: if prefix with `%`, allow use a keyword as `ident`.

When testing it, `aa + bb` is parsed succefully, `x + y` will failed because `x` is a keyword, not a `ident`, as excepted.

Next it parse `%x + y`, and panic.

If we change pattern of `ident` to `[[:alpha:]][[:alnum:]]*` (Just remove the inner group, or add `?:` make it non-capture), it will works fine.

Test code

package main

import (
"fmt"

"github.com/alecthomas/participle/v2"
"github.com/alecthomas/participle/v2/lexer"
)

type Plus struct {
Lhs string `parser:"@Ident"`
Op string `parser:"@'+'"`
Rhs string `parser:"@Ident"`
}

func main() {
parser := participle.MustBuild[Plus](
participle.Lexer(lexer.MustStateful(lexer.Rules{
"Root": {
{"whitespace", ` +`, nil},
{"Op", `\+`, nil},
{"Keyword", `x`, nil},
{"Ident", `[[:alpha:]]([[:alnum:]])*`, nil},
{"percent", `%`, lexer.Push("Percent")},
},
"Percent": {
{"Ident", `[[:alpha:]]([[:alnum:]])*`, lexer.Pop()},
},
})),
)

ast, err := parser.ParseString("input", "aa + bb")
fmt.Printf("ast: %#v, err: %#v\n\n", ast, err)

ast, err = parser.ParseString("input", "x + y")
fmt.Printf("ast: %#v, err: %#v\n\n", ast, err)

ast, err = parser.ParseString("input", "%x + y")
fmt.Printf("ast: %#v, err: %#v\n\n", ast, err)
}

Maybe caused by [here](https://github.com/alecthomas/participle/blob/27540101d8610a60b457f800b55de8205c5f9977/lexer/stateful.go#L387), the pattern of `Ident` call `FindStringSubmatchIndex` with data `x + y` will return a `[0, 1, -1, -1]`, the last two `-1` means the inner group `([[:alnum:]])` never captures:

![](https://user-images.githubusercontent.com/7822577/225391947-8575d14b-bc9a-4a71-8dc4-a76f08ae454c.png)

The simplest way to fixes this may be add a guard:
````golang
if match[i] >= 0 {
groups = append(groups, l.data[match[i]:match[i+1]])
}
````

But I'm not sure how this group intergrate with Action interface, may be it needs some type to indicate the empty capture? So I just open this issue, not a PR.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.