oxc-project / oxc-project/backlog

Maintain cache coherence in transformer

Open
#34 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
No language data
Stars
7
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Boshen mentioned this is meeting yesterday. Oxc's AST has the advantage of storing all nodes in arena in traversal order (broadly speaking - Vec content will jump around when Vecs grow (#35)).

The transformer throws the distribution of AST nodes in memory into disarray because new nodes it inserts are stored at the end (bottom) of the arena, far away from the AST data for surrounding nodes.

I do not think this problem is solvable in general. However, we could ameliorate it somewhat for JSX and TS syntax where it's predictable how it's going to be transformed. For example:

<Foo />

is transformed to:

_jsx(Foo, {})

We could make it so when parser writes a JSXElement into arena, it leaves sufficient free bytes after it so that the JSXElement + unused bytes is enough bytes to store a CallExpression, Vec<Argument>, ObjectExpression, and Vec<ObjectPropertyKind>. The transformer could use that space to replace the JSXElement in place in the arena.

There are 3 downsides:

  1. Hard to calculate how much space is needed ahead of time (depends on whether classic/automatic mode JSX transform).
  2. Risk of overwriting other AST nodes (which would be UB).
  3. Currently we know that a *mut Whatever pointing into arena always points to valid data until the arena is reset(), because even if the node is removed from AST, the old node still sits in same place in the arena. Replacing nodes in place would break this invariant. It may make unsafe code e.g. Traverse harder to make sound.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or concrete entry points are named. Start by tracing the parser's AST arena, JSX and TypeScript transformer paths, and the Traverse raw-pointer invariant; define completion as a measured coherence improvement that does not introduce arena overwrites or invalidate pointer safety.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
compilers, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.