MemberJunction / MemberJunction/MJ

Split generated entity/GraphQL monoliths + enable incremental tsc

Open
#3,784 0 comments 0 reactions 2 assignees Claimed by @rkihm-BC View on GitHub
enhancement priority: medium
Dominant language
TSQL
Stars
29
Forks
6
Avg merge
1d 8h
Merged PRs (30d)
308

Description

## Why this exists

A performance pass on the repo flagged the **generated output as one-file monoliths** as a first-class developer-loop cost. PR #3780 removes the unused GraphQL child-array FieldResolvers (~10k lines off `generated.ts`) — that helps schema size and kills an N+1 footgun, but it barely moves the **compile** bill.

@cadam11 @rkihm-BC — please review. If you concur, this is yours to implement (or to reject / reshape). I am not in this wheelhouse day-to-day; the write-up is the recommendation, not a mandate.

## The actual problem

Turbo caches **packages**, not files. CodeGen currently concatenates every entity into one file, then writes it:

| Artifact | Approx size | Why it hurts |
|---|---|---|
| `packages/MJCoreEntities/src/generated/entity_subclasses.ts` | ~120k lines (Zod + classes) | One field change rewrites the whole file → `@memberjunction/core-entities` cache miss → half the monorepo rebuilds. Almost everything imports this package. |
| `packages/MJServer/src/generated/generated.ts` | ~86k after #3780 | Same package-cache story, **plus** it lives in the same `tsc` project as hand-written resolvers (`UserResolver`, etc.). A one-line resolver fix recompiles the monolith. `buildSchemaSync` is a **separate** O(types × fields) cost at API boot and is **not** fixed by splitting files. |

Angular already sharded this (`maxComponentsPerModule: 25` in CodeGen). Entity subclasses and GraphQL server types did not.

CodeGen already walks entities one at a time, then concatenates. Entity subclasses:

```ts
// entity_subclasses_codegen.ts generateAllEntitySubClasses
const zodContent = entities.map(e => this.GenerateSchemaAndType(e)).join('');
// ... generateEntitySubClass per entity ...
fs.writeFileSync(..., 'entity_subclasses.ts', allContent);
```

GraphQL already has `generateEntitySpecificServerFileHeader` and then writes **one** `generated.ts`.

## What will not help (do not do these for the wrong reason)

- **Subpath exports so Explorer imports one entity.** `@RegisterClass` + the class-registration manifest exist because those classes cannot be tree-shaken. You will still load all of them at startup.
- **Hand-splitting `generated.ts` / `entity_subclasses.ts` in a one-off PR.** Next `mj codegen` glues them back. The emitter has to own the layout.
- **Splitting GraphQL files to make `buildSchemaSync` faster.** TypeGraphQL still reflects every `@ObjectType`. Schema-build is a different problem (metadata hash → skip rebuild, or fewer types in the schema). #3780 was the right *schema* cut.

## Recommended approach (in order)

### 1. Turn on `incremental` (and later `composite`) — tiny PR, do first

`tsconfig.server.json` has neither. A `.tsbuildinfo` on `@memberjunction/core-entities` and `@memberjunction/server` is the cheapest win on "I changed one hand-written file next to a monolith." Works *today* with the single files. No CodeGen rewrite.

### 2. Per-entity emit + skip-if-identical write

Change CodeGen to emit:

- `generated/entities/MJAction.ts` (Zod + class)
- `generated/entities/MJUser.ts`
- `generated/entity_subclasses.ts` as a barrel `export * from './entities/…'`

Same shape for GraphQL server types if you want incremental `tsc` of MJServer.

**Do not write the file if the bytes did not change.** Today one new column rewrites 120k lines, so git, mtime, and turbo all see a full-package change even for entities that did not move.

Public import surface stays `@memberjunction/core-entities`. Callers do not change.

This is the CodeGen change that makes incremental `tsc` actually fire after a one-entity schema change.

### 3. Do not put generated GraphQL in the same package as hand-written resolvers

A generated-only package (e.g. `@memberjunction/server-generated`) or a project-reference tsconfig so:

- entity / CodeGen change → rebuild the generated project
- `UserResolver` change → rebuild the thin hand-written project

That is a Turbo-shaped win because the cache unit becomes two packages instead of one.

### 4. Leave schema-build as a follow-up

If API boot at scale is still ~66s in `buildSchemaSync`, that wants a schema cache (hash of entity metadata → skip `buildSchemaSync`) or fewer registered types — not more files.

## Expected payoff

| Change | What gets faster |
|---|---|
| `incremental` only | Repeat `tsc` of core-entities / MJServer when generated output did not change |
| Per-entity + skip-identical | CodeGen write, git diffs, incremental emit after a one-entity schema change |
| Generated GraphQL as its own project | Hand-written server edits stop paying the 86k-line typecheck |

## Out of scope for the first PR

- Zod in a separate package (`import type` for tests) — nice later, not the bottleneck.
- Changing `@RegisterClass` / the manifest.
- OpenApp codegen layout beyond "the same emitter owns per-entity files."

## Context

- Performance assessment that produced this: conversation on the generated-output / CodeGen loop (2026-08-13).
- Adjacent shipped work: #3780 (stop emitting `*Array` FieldResolvers).
- Precedent: Angular form codegen already shards with `maxComponentsPerModule`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.