An error occurred when antlr-parse is decoding the output.
- Dominant language
- Java
- Stars
- 19k
- Forks
- 3.5k
- PR merge metrics
- No merged PRs in 30d
Description
When using antlr-parse to parse a file in utf-8 with encoding set to utf-8, an error occurred like below:
```text
Traceback (most recent call last): File "D:\scoop\apps\python39\current\lib\runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "D:\scoop\apps\python39\current\lib\runpy.py", line 87, in _run_code exec(code, run_globals) File "D:\scoop\apps\python39\current\Scripts\antlr4-parse.exe\__main__.py", line 7, in File "D:\scoop\apps\python39\current\lib\site-packages\antlr4_tool_runner.py", line 153, in interp err = err.decode("UTF-8")UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 198: invalid continuation byte
```
It seems the actual encoding of output is *gbk*, while it's decoded with *utf-8*. I've tried *chcp*(I'm using Windows), but the error remains.
An example grammar file is
```antlr4
grammar test;
// Comment: '//'~[\n\r]* -> skip;
EmptyLine: [\n\r]+ -> skip;
Space: [ \t] -> skip;
Token: ~[ \t\n\r]+;
program: Token*;
```
An example input file is
```text
任意的 Unicode 字符
```
Command is
```cmd
antlr4-parse test.g4 program -tree -encoding utf-8 test.txt
```
Contributor guide
Research direction
Start with antlr4_tool_runner.py at the interp path reported at line 153, then reproduce the command `antlr4-parse test.g4 program -tree -encoding utf-8 test.txt` on Windows using the supplied grammar and Unicode input. Trace how the command output is decoded and verify that the error no longer raises UnicodeDecodeError for the example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100