MemberJunction / MemberJunction/MJ
CodeGen: a single failed Application create locks a new schema out permanently (isSchemaNew latch)
- Dominant language
- TSQL
- Stars
- 29
- Forks
- 6
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 323
Description
Split out of the review on #3932, at @rkihm-BC's request. #3932 fixes the PostgreSQL quoting bug that made `createNewApplication` fail on every run; this issue is the *design* flaw that made that failure — or any other — permanent. It is independent of the quoting bug and is not PostgreSQL-specific.
## The defect
`createNewEntity` gates Application creation on `isSchemaNew()`, and that gate is a one-shot latch:
```ts
// packages/CodeGenLib/src/Database/manage-metadata.ts:5902
const isNewSchema = await this.isSchemaNew(pool, newEntity.SchemaName);
const newEntityID = this.createNewUUID();
await this.LogSQLAndExecute(pool, sSQLInsert, ...); // entity row commits here, unconditionally
// :5915
if (isNewSchema && configInfo.newSchemaDefaults.CreateNewApplicationWithSchemaName) {
apps = await this.getApplicationIDForSchema(...);
if (!apps || apps.length === 0) {
const newAppID = await this.createNewApplication(...);
if (newAppID) { apps = [newAppID]; }
else { LogError(` >>>> ERROR: Unable to create new application for schema ${newEntity.SchemaName}`); }
}
} else {
apps = await this.getApplicationIDForSchema(pool, newEntity.SchemaName); // pure lookup, no create path
}
```
Chain of events:
1. `isSchemaNew()` (`:5992`) is `SELECT COUNT(*) FROM __mj.Entity WHERE SchemaName = X` against **live database state** — not an in-process flag scoped to the current CodeGen run.
2. It is evaluated **before** the entity INSERT, and that INSERT commits unconditionally, regardless of what happens next.
3. If `createNewApplication` fails for **any** reason — a deadlock, a permissions problem, a transient connection error, or (pre-#3932, always) the PG quoting bug — the schema gets no Application. The entity row is already committed.
4. On entity #2 of that schema — in this run or any future run — `isSchemaNew()` returns `false`, because entity #1 is permanently in `__mj.Entity`. The `else` branch calls only `getApplicationIDForSchema` (`:6098`), which is a pure `SELECT ... FROM vwApplications` with no create path.
5. There is no reconciliation sweep elsewhere: grepping `CreateNewApplicationWithSchemaName` / `SchemaAutoAddNewEntities` across CodeGenLib turns up nothing that retroactively creates a missing Application.
**Net effect:** one transient failure on entity #1 of a new schema means that schema can never get an Application through CodeGen again, for the lifetime of the database. Every entity in it renders in the UI's "System & Other" bucket. Tables and data are unaffected — only the grouping is wrong — and CodeGen exits 0 throughout, so nothing surfaces it.
## The fix
The shared helper `addEntityToApplicationForSchema` (`:6126`) already does the right thing: it looks up and creates-if-absent on **every** call, so a failure on entity #1 is simply retried on entity #2. It is already the path used for virtual entities (`:1339`) and query entities (`:1777`).
Replace `createNewEntity`'s bespoke `isSchemaNew()`-gated block (`:5902`, `:5915-5951`) with a call to that helper, matching the pattern the other two call sites already use. That also deletes the now-unused `isSchemaNew()` and removes a duplicated `ApplicationEntity` INSERT.
Worth checking as part of the change:
- `isSchemaNew()` has no other callers before removing it.
- The helper's `md.Refresh()` placement matches the current behaviour closely enough for the table-backed path.
- The helper logs a `WARNING` where the current block logs an `ERROR` when no application is found — decide which level is right for the merged path.
## Repro
On either platform, with `CreateNewApplicationWithSchemaName: true`:
1. Point CodeGen at a schema with ≥2 new tables.
2. Make the `Application` INSERT fail on the first entity (revoke INSERT on `__mj.Application`, or check out any commit before #3932 and run on PostgreSQL).
3. Restore the condition and re-run CodeGen any number of times.
Expected: the schema eventually gets its Application. Actual: it never does — `isSchemaNew()` is now `false` forever.
## Scope
`packages/CodeGenLib/src/Database/manage-metadata.ts` only. No migration, no metadata, no schema change. Patch-level.
Contributor guide
Research direction
Start in packages/CodeGenLib/src/Database/manage-metadata.ts at createNewEntity, then compare the existing addEntityToApplicationForSchema calls at lines 1339, 1777, and 6126. Check whether isSchemaNew has other callers and inspect the helper's md.Refresh() and logging behavior. Done means a failed first Application creation can be retried for later entities or later CodeGen runs, with the relevant CodeGen reproduction passing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend, database
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100