needs_separator_when_before omits CDC, so serialized tokens re-parse wrong
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 869
- Forks
- 152
- Avg merge
- 15h 8m
- Merged PRs (30d)
- 12
Description
TokenSerializationType::needs_separator_when_before returns false when a Number, DelimHash, DelimAt, or DelimMinus is followed by CDC. Concatenating the two serializations produces text that tokenizes into different tokens than the input.
Reproducer
cssparser 0.37.0, default features.
use cssparser::{Parser, ParserInput, ToCss};
fn main() {
let mut pi = ParserInput::new("5 -->");
let mut p = Parser::new(&mut pi);
let a = p.next().unwrap().clone(); // Number { value: 5.0, int_value: Some(5) }
let b = p.next().unwrap().clone(); // CDC
// no separator requested
assert!(!a
.serialization_type()
.needs_separator_when_before(b.serialization_type()));
let mut s = String::new();
a.to_css(&mut s).unwrap();
b.to_css(&mut s).unwrap();
assert_eq!(s, "5-->");
}
Observed vs expected
Observed: the call returns false and serialization yields 5-->. Re-parsing 5--> gives Dimension { value: 5.0, unit: "--" } followed by Delim('>').
Expected: true, which signals that an empty comment is needed so the output round-trips. The doc comment on needs_separator_when_before says it returns true if "an empty comment /**/ needs to be inserted between them so that they are not re-parsed as a single token". CSS Syntax Level 3 §Serialization requires the serialized form to round-trip.
Root cause
src/serializer.rs:519. The Ident and AtKeywordOrHash | Dimension rows list CDC. The rows covering Number, DelimHash, DelimAt, and DelimMinus do not.
Scope
Four pairs return false: Number then CDC gives 5-->, re-parsing as Dimension{5,"--"} + Delim('>'). DelimHash then CDC gives #-->, re-parsing as IDHash("--") + Delim('>'). DelimAt then CDC gives @-->, re-parsing as AtKeyword("--") + Delim('>'). DelimMinus then CDC gives --->, re-parsing as Ident("---") + Delim('>').
Any consumer using this API to re-emit CSS is affected. Stylo calls it for custom-property and var() substitution serialization. Present on main as of the 2026-07-21 commit.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/serializer.rs around line 519 and read needs_separator_when_before, especially the rows for Number, DelimHash, DelimAt, and DelimMinus. Verify the four CDC combinations against the issue's reproducers and existing serializer coverage. Done means each combination requests a separator and serialized tokens re-parse to the original token sequence.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100