MemberJunction / MemberJunction/MJ

CodeGen: a single failed Application create locks a new schema out permanently (isSchemaNew latch)

Open
#4,034 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
TSQL
Stars
29
Forks
6
Avg merge
2d 1h
Merged PRs (30d)
323

Description

Split out of the review on #3932, at @rkihm-BC's request. #3932 fixes the PostgreSQL quoting bug that made `createNewApplication` fail on every run; this issue is the *design* flaw that made that failure — or any other — permanent. It is independent of the quoting bug and is not PostgreSQL-specific.

## The defect

`createNewEntity` gates Application creation on `isSchemaNew()`, and that gate is a one-shot latch:

```ts
// packages/CodeGenLib/src/Database/manage-metadata.ts:5902
const isNewSchema = await this.isSchemaNew(pool, newEntity.SchemaName);
const newEntityID = this.createNewUUID();
await this.LogSQLAndExecute(pool, sSQLInsert, ...); // entity row commits here, unconditionally

// :5915
if (isNewSchema && configInfo.newSchemaDefaults.CreateNewApplicationWithSchemaName) {
apps = await this.getApplicationIDForSchema(...);
if (!apps || apps.length === 0) {
const newAppID = await this.createNewApplication(...);
if (newAppID) { apps = [newAppID]; }
else { LogError(` >>>> ERROR: Unable to create new application for schema ${newEntity.SchemaName}`); }
}
} else {
apps = await this.getApplicationIDForSchema(pool, newEntity.SchemaName); // pure lookup, no create path
}
```

Chain of events:

1. `isSchemaNew()` (`:5992`) is `SELECT COUNT(*) FROM __mj.Entity WHERE SchemaName = X` against **live database state** — not an in-process flag scoped to the current CodeGen run.
2. It is evaluated **before** the entity INSERT, and that INSERT commits unconditionally, regardless of what happens next.
3. If `createNewApplication` fails for **any** reason — a deadlock, a permissions problem, a transient connection error, or (pre-#3932, always) the PG quoting bug — the schema gets no Application. The entity row is already committed.
4. On entity #2 of that schema — in this run or any future run — `isSchemaNew()` returns `false`, because entity #1 is permanently in `__mj.Entity`. The `else` branch calls only `getApplicationIDForSchema` (`:6098`), which is a pure `SELECT ... FROM vwApplications` with no create path.
5. There is no reconciliation sweep elsewhere: grepping `CreateNewApplicationWithSchemaName` / `SchemaAutoAddNewEntities` across CodeGenLib turns up nothing that retroactively creates a missing Application.

**Net effect:** one transient failure on entity #1 of a new schema means that schema can never get an Application through CodeGen again, for the lifetime of the database. Every entity in it renders in the UI's "System & Other" bucket. Tables and data are unaffected — only the grouping is wrong — and CodeGen exits 0 throughout, so nothing surfaces it.

## The fix

The shared helper `addEntityToApplicationForSchema` (`:6126`) already does the right thing: it looks up and creates-if-absent on **every** call, so a failure on entity #1 is simply retried on entity #2. It is already the path used for virtual entities (`:1339`) and query entities (`:1777`).

Replace `createNewEntity`'s bespoke `isSchemaNew()`-gated block (`:5902`, `:5915-5951`) with a call to that helper, matching the pattern the other two call sites already use. That also deletes the now-unused `isSchemaNew()` and removes a duplicated `ApplicationEntity` INSERT.

Worth checking as part of the change:
- `isSchemaNew()` has no other callers before removing it.
- The helper's `md.Refresh()` placement matches the current behaviour closely enough for the table-backed path.
- The helper logs a `WARNING` where the current block logs an `ERROR` when no application is found — decide which level is right for the merged path.

## Repro

On either platform, with `CreateNewApplicationWithSchemaName: true`:

1. Point CodeGen at a schema with ≥2 new tables.
2. Make the `Application` INSERT fail on the first entity (revoke INSERT on `__mj.Application`, or check out any commit before #3932 and run on PostgreSQL).
3. Restore the condition and re-run CodeGen any number of times.

Expected: the schema eventually gets its Application. Actual: it never does — `isSchemaNew()` is now `false` forever.

## Scope

`packages/CodeGenLib/src/Database/manage-metadata.ts` only. No migration, no metadata, no schema change. Patch-level.

Contributor guide

Open the contributing guide

Research direction

Start in packages/CodeGenLib/src/Database/manage-metadata.ts at createNewEntity, then compare the existing addEntityToApplicationForSchema calls at lines 1339, 1777, and 6126. Check whether isSchemaNew has other callers and inspect the helper's md.Refresh() and logging behavior. Done means a failed first Application creation can be retried for later entities or later CodeGen runs, with the relevant CodeGen reproduction passing.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, database
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.