antlr / antlr/antlr4

[Feature] Parsing of binary streams instead of utf-8?

Open
#3,532 1 comment 0 reactions 0 assignees View on GitHub
type:question unicode
Dominant language
Java
Stars
19k
Forks
3.5k
PR merge metrics
No merged PRs in 30d

Description

ANTLR should support the parsing of raw binary streams (1 byte per input character) alongside UTF-8 characters.

have it available as an encoding, for example:
`antlr4 -encoding raw Grammar.g4`

For example to match the byte `0x80` it is not possible right now as `\u0080` evaluates to `0xC280`

Contributor guide

Open the contributing guide

Research direction

Start by reviewing ANTLR's input-stream handling and the command-line entry point for the proposed `-encoding raw` option. Define the runtime scope and tests needed to verify that raw bytes, including 0x80, can be matched without UTF-8 conversion; done means binary-stream parsing works alongside UTF-8 parsing.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
compilers
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.