oxc-project / oxc-project/backlog

AST: Remove `raw` field from `StringLiteral`

Open
#229 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
No language data
Stars
7
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Summary

Remove StringLiteral::raw from the Rust AST.

value is the semantic string value, while the original token spelling is already available from the node's span and the source text owned by parser-origin workflows. Keeping both on the node duplicates source data and permits them to disagree.

Motivation

Several source spellings can have the same value:

"value"
'value'
"\x76alue"
"\u0076alue"

The AST currently stores both the decoded value and an optional raw token string. Their relationship is not enforced. Builders and transforms can create states such as:

  • value == "a" with raw == Some("\"b\"");
  • a mutated value with stale source spelling in raw;
  • a parser-origin node with raw: None;
  • a synthetic node with an arbitrary raw unrelated to its span.

Most semantic consumers need only value. Source-aware consumers such as formatters, diagnostics, lint fixes, and ESTree serialization already have the source text and can slice span. Synthetic nodes have no original spelling to preserve; their printers should quote and escape value.

Removing the field also removes one arena string reference from every string literal, which is significant because string literals occur throughout JavaScript, TypeScript, JSX attributes, module specifiers, and property keys.

Current AST Shape

pub struct StringLiteral<'a> {
    pub node_id: Cell<NodeId>,
    pub span: Span,
    #[estree(via = StringLiteralValue)]
    pub value: Str<'a>,
    #[content_eq(skip)]
    #[estree(from_span)]
    pub raw: Option<Str<'a>>,
    #[builder(default)]
    #[estree(skip)]
    pub lone_surrogates: bool,
}

raw is skipped by content equality and serialized from source spans for ESTree, which already acknowledges that it is source metadata rather than semantic AST state.

Proposed AST Shape

pub struct StringLiteral<'a> {
    pub node_id: Cell<NodeId>,
    pub span: Span,
    #[estree(via = StringLiteralValue)]
    pub value: Str<'a>,
    #[builder(default)]
    #[estree(skip)]
    pub lone_surrogates: bool,
}

Source-aware APIs should use a helper equivalent to:

fn raw_string_literal<'s>(literal: &StringLiteral<'_>, source_text: &'s str) -> &'s str {
    &source_text[literal.span.start as usize..literal.span.end as usize]
}

Real code should use the repository's checked span/source helpers rather than unchecked indexing. ESTree serialization can continue emitting raw from the serializer's source context. Codegen for synthetic or modified nodes should escape value according to its output options.

Syntax Mapping

Source form Current representation Proposed representation
"a" value: "a", raw: Some("\"a\"") value: "a"; source spelling available through span
'a' value: "a", raw: Some("'a'") Same AST value as "a"; source context preserves quote choice
"\x61" value: "a", escaped raw value: "a"; source context preserves escape spelling
Synthetic string value, usually raw: None value; printer chooses a valid spelling

Guaranteed Invariants

  • A string literal has one semantic string value.
  • The AST cannot contain a raw spelling that contradicts value.
  • Parser-origin source spelling has one authority: source text plus span.
  • Synthetic strings do not pretend to have an original token spelling.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the Rust AST StringLiteral definition and every use of its raw field, including builders, transforms, ESTree serialization, formatters, diagnostics, lint fixes, and codegen. Done means the field and its related metadata are removed, source-aware consumers use span and source text, and synthetic nodes still print valid values according to output options.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
compilers
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.