HexHive / HexHive/Gramatron

Error in generalizing a terminal into a nonterminal in the gnf_converter script

Open
#4 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
80
Forks
7
PR merge metrics
No merged PRs in 30d

Description

Hi,
looking at the quality of the generated testcases using the Ruby grammar, I noticed that my implementation of gramatron in LibAFL was generating invalid testcases with lines like IDENTIFIER SP VAR.
Debugging further, I noticed that the error is in the original Python script that I borrowed from this repo, gnf_converter.py.

Here https://github.com/HexHive/Gramatron/blob/02ab42d857ff2d3bc8c872f4756e6af9e3baaf0f/src/gramfuzz-mutator/preprocess/gnf_converter.py#L198 you check if a terminal can be replaced with a nonterminal, but this is wrong in case of a terminal repeated two times in two different nonterminals of the grammar.

For this specific case, I noticed the bug because in the Ruby grammar 'return' is both in IDENTIFIER https://github.com/HexHive/Gramatron/blob/main/grammars/ruby/source.json#L843 and in STATEMENT https://github.com/HexHive/Gramatron/blob/main/grammars/ruby/source.json#L1169

Ofc replacing 'return' SP VAR with IDENTIFIER SP VAR is wrong. I fixed it just removing the call to ` terminal_exist` but dunno if it is 100% right so I won't open a PR.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in src/gramfuzz-mutator/preprocess/gnf_converter.py around line 198, then compare the repeated `return` entries in grammars/ruby/source.json at lines 843 and 1169. Run the converter with the Ruby grammar and inspect generated cases involving `return` SP VAR. Done means the converter no longer produces the invalid `IDENTIFIER SP VAR` substitution for this repeated terminal case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, ruby
Domain
testing-qa, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.