servo / servo/rust-cssparser

needs_separator_when_before omits CDC, so serialized tokens re-parse wrong

Open Beginner friendly
#434 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
869
Forks
152
Avg merge
15h 8m
Merged PRs (30d)
12

Description

TokenSerializationType::needs_separator_when_before returns false when a Number, DelimHash, DelimAt, or DelimMinus is followed by CDC. Concatenating the two serializations produces text that tokenizes into different tokens than the input.

Reproducer

cssparser 0.37.0, default features.

use cssparser::{Parser, ParserInput, ToCss};

fn main() {
    let mut pi = ParserInput::new("5 -->");
    let mut p = Parser::new(&mut pi);
    let a = p.next().unwrap().clone(); // Number { value: 5.0, int_value: Some(5) }
    let b = p.next().unwrap().clone(); // CDC

    // no separator requested
    assert!(!a
        .serialization_type()
        .needs_separator_when_before(b.serialization_type()));

    let mut s = String::new();
    a.to_css(&mut s).unwrap();
    b.to_css(&mut s).unwrap();
    assert_eq!(s, "5-->");
}

Observed vs expected

Observed: the call returns false and serialization yields 5-->. Re-parsing 5--> gives Dimension { value: 5.0, unit: "--" } followed by Delim('>').

Expected: true, which signals that an empty comment is needed so the output round-trips. The doc comment on needs_separator_when_before says it returns true if "an empty comment /**/ needs to be inserted between them so that they are not re-parsed as a single token". CSS Syntax Level 3 §Serialization requires the serialized form to round-trip.

Root cause

src/serializer.rs:519. The Ident and AtKeywordOrHash | Dimension rows list CDC. The rows covering Number, DelimHash, DelimAt, and DelimMinus do not.

Scope

Four pairs return false: Number then CDC gives 5-->, re-parsing as Dimension{5,"--"} + Delim('>'). DelimHash then CDC gives #-->, re-parsing as IDHash("--") + Delim('>'). DelimAt then CDC gives @-->, re-parsing as AtKeyword("--") + Delim('>'). DelimMinus then CDC gives --->, re-parsing as Ident("---") + Delim('>').

Any consumer using this API to re-emit CSS is affected. Stylo calls it for custom-property and var() substitution serialization. Present on main as of the 2026-07-21 commit.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/serializer.rs around line 519 and read needs_separator_when_before, especially the rows for Number, DelimHash, DelimAt, and DelimMinus. Verify the four CDC combinations against the issue's reproducers and existing serializer coverage. Done means each combination requests a separator and serialized tokens re-parse to the original token sequence.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
tooling
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.