nodejs / nodejs/orchestrion-js

Transformer walks the AST once per config instead of once per file, worth optimizing?

Open
#83 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
24
Forks
11
PR merge metrics
No merged PRs in 30d

Description

Problem Statement

I was working on one of our instrumentation refactors, and I ended up having dozens of configs (via @apm-js-collab/tracing-hooks) per one source code file. Ultimately, that wasn't the best way of doing things, but it did reveal that Transformer.transform walks the AST once per config, resulting in N traversals of the same AST for a file.

I am curious if this is an observation other APM vendors have made and that if the following optimization would be worth it. Essentially, the idea is that we walk the AST once per file rather than once per config.

Potential Solution

Walk the AST once for all configs. Collect every config's selector up front, traverse the union of them a single time, and at each matched node dispatch to each config whose own selector matches:

const matchers = [] // { state, query, selector } per config

for (const config of this.#configs) {
  // ...build query/state as before...
  matchers.push({ state, query, selector: esquery.parse(query) })
}

if (matchers.length > 0) {
  const combined = esquery.parse(matchers.map(({ query }) => query).join(', '))
  esquery.traverse(ast, combined, (node, parent, ancestry) => {
    for (const { state, selector } of matchers) {
      if (esquery.matches(node, selector, ancestry)) {
        injectionCount++
        this.#visit(state, node, parent, ancestry)
      }
    }
  })
}

This takes traversals from N down to 1, regardless of how many configs target the file. But I think this is a constant-factor win — total matching work stays O(configs × AST nodes), since at each node we still check it against every config's selector. What we actually save is running esquery's tree-walk machinery once instead of once per config. So I'm curious whether that overhead is meaningful enough in practice to be worth it, or if I'm just working around a config setup I shouldn't have had in the first place.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Transformer.transform in lib/transformer.js, especially lines 81-119, and trace how configs build queries and invoke esquery traversal. Measure the current behavior with many configs on one source file, then evaluate whether a single combined traversal provides a meaningful improvement without changing matching or injection behavior. Done means a supported optimization decision backed by measurements and any necessary regression coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.