is_ident_start reads the wrong byte for a hyphen before a backslash
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 869
- Forks
- 152
- Avg merge
- 15h 8m
- Merged PRs (30d)
- 12
Description
When a - is followed by a backslash that begins an escape, the check that decides whether an identifier starts here reads the wrong byte. A backslash right before a newline is an invalid escape and cannot start an identifier, so -\<newline> is mis-tokenized.
Reproducer
cssparser 0.37.0, default features.
use cssparser::{Parser, ParserInput};
fn main() {
let mut pi = ParserInput::new("1-\\\n");
let mut p = Parser::new(&mut pi);
let a = p.next().unwrap().clone();
// cssparser: Dimension { value: 1.0, int_value: Some(1), unit: "-" }
println!("{:?}", a);
}
Observed vs expected
Observed for 1-\<newline>: the tokens are Dimension { value: 1.0, unit: "-" } then Delim('\\'). For -\<newline> the first token is Ident("-"). For #-\<newline> the first token is IDHash("-").
Expected per CSS Syntax Level 3, matched by tinycss2: 1-\<newline> gives Number(1), Delim('-'), Delim('\\'). -\<newline> gives Delim('-') then Delim('\\'). #-\<newline> gives a hash token Hash("-") with the unrestricted type flag, not an id hash.
Root cause
src/tokenizer.rs:895. The would-start-an-identifier test reads the byte after the backslash to decide if -\ begins an identifier. A backslash directly before a newline is not a valid escape, but the offset used skips the backslash and reads the newline as though it were escaped.
Scope
A hyphen before an invalid escape produces a unit, an identifier, or an id hash that the input never held. A consumer that keys on token type, such as a linter or a minifier, then acts on tokens the source did not contain.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at src/tokenizer.rs:895 and run the Rust reproducer against the default features. Check the identifier-start decision for a hyphen before a backslash-newline sequence, then verify that the three reported inputs produce the expected token types and values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- css, rust
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 88/100