overengineeringstudio / overengineeringstudio/effect-utils

genie --check is a bottleneck in dt runs (~1.5s per invocation)

Open
#235 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

origin:agent type:agent-tooling
Dominant language
TypeScript
Stars
82
Forks
2
Avg merge
1d 8h
Merged PRs (30d)
121

Description

Problem

genie --check takes ~1.5s per invocation and is called twice per dt run due to devenv DAG re-evaluation, adding ~3s to every shell entry.

Observed in trace f54fe17cd7e16bbf574f00e3f687a950 from a cold dt megarepo:sync run (1114 spans total, 938 from genie).

Timing breakdown (single invocation, ~1.5s)

├── Bun startup + CLI parse + OTEL init:  ~420ms (28%)  [fixed overhead]
├── genie/checkAll:                       ~956ms (65%)
│   ├── discoverAndValidate:               ~67ms  (find 65 .genie.ts files)
│   ├── checkFile x65 (parallel):         ~564ms
│   │   ├── formatWithOxfmt:  311ms avg   ← DOMINANT BOTTLENECK
│   │   ├── findRepoRoot:    141ms avg    ← redundant (same root, computed 65x)
│   │   └── import():         19ms avg
│   └── runValidationOrFail:             ~321ms
│       ├── findGenieFiles (AGAIN):        ~48ms  ← redundant re-discovery
│       └── workspace resolution:         ~160ms
└── OTEL shutdown flush:                   ~56ms  (4%)

Bottlenecks ranked

  1. formatWithOxfmt (311ms avg/file) — spawns 65 oxfmt subprocesses concurrently (concurrency: 'unbounded'). Native oxfmt is ~30-50ms but OS contention from 65 concurrent spawns inflates each to 311ms. Total CPU: 40s.

  2. findRepoRoot (141ms avg/file) — walks up directory tree independently for all 65 files. Same result every time, not cached.

  3. Bun startup (~420ms) — fixed per-invocation overhead, makes binary status checks inherently expensive.

  4. Redundant findGenieFiles — called in discoverAndValidate, then again in runValidationOrFail. Same 65 files discovered twice.

  5. Double invocation — devenv calls genie:run:status twice per dt run due to DAG re-evaluation passes.

Improvement ideas

Idea Effort Estimated saving
Switch status to "hash" (skip binary entirely) Medium ~1.5s/invocation
Limit formatWithOxfmt concurrency to 8-16 Low ~200ms/invocation
Cache findRepoRoot (compute once, share) Low ~140ms/invocation
Share discovery results with validation Low ~200ms/invocation
Skip oxfmt in --check mode (compare pre-format) Medium ~500ms/invocation

Switching status from "binary" to "hash" would be the biggest single win — comparing file hashes in bash instead of spawning Bun + genie + oxfmt drops status checks from ~1.5s to ~10ms.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the genie --check entry point and trace checkAll, discoverAndValidate, checkFile, formatWithOxfmt, findRepoRoot, and runValidationOrFail. Compare the timing breakdown with a cold dt megarepo:sync run, then select and validate an optimization from the listed ideas. Done means the chosen change reduces status-check time without changing validation behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
bun, typescript
Domain
cli, performance, tooling
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.