MemberJunction / MemberJunction/MJ

PostgreSQL: five core EntityFields are declared in metadata but missing from their base views

Open
#3,869 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TSQL
Stars
29
Forks
6
Avg merge
1d 8h
Merged PRs (30d)
308

Description

## One sentence

On PostgreSQL, five core `EntityField`s are declared in metadata and exist as real columns on their base tables, but are **not exposed by their base views** — so metadata promises a field the view cannot serve.

## The audit

Fresh PostgreSQL 17.10 database, `mj migrate --tag v6.1.0-edge.2`, nothing else applied. Cross-referencing MJ's own metadata against the catalog:

```sql
SELECT e."BaseTable" || '.' || f."Name",
CASE WHEN EXISTS (SELECT 1 FROM information_schema.columns vc
WHERE vc.table_schema='__mj' AND vc.table_name=e."BaseView" AND vc.column_name=f."Name")
THEN 'in view' ELSE 'MISSING from view' END
FROM __mj."EntityField" f
JOIN __mj."Entity" e ON e."ID" = f."EntityID"
WHERE e."SchemaName" = '__mj';
```

| Field | IsVirtual | AllowUpdateAPI | on base table | in base view |
|---|---|---|---|---|
| `Entity.AllowDirectSQLDelete` | false | true | yes | **no** |
| `Entity.AllowDirectSQLInsert` | false | true | yes | **no** |
| `Entity.AllowDirectSQLUpdate` | false | true | yes | **no** |
| `Entity.GeneratedBaseViewName` | false | true | yes | **no** |
| `EntityRelationship.RelatedRecordCollection` | false | true | yes | **no** |

None are virtual, all are updatable through the API, and all exist on the base table. The view is simply missing them.

This is an internal inconsistency rather than a judgement call: the metadata layer says the field exists, so anything reading the entity expects the view to return it.

## Not everything absent from a view is a bug — this is the filtered list

`Entity.ExternalDataSourceID` and `Entity.ExternalObjectName` are absent from `vwEntities` too, and are **correctly** absent: they carry no `EntityField` row on `Entity`, consistent with #3514's note that `vwEntities.ExternalDataSourceID` was deliberately dropped. Filtering on "declared as an EntityField" is what separates the five above from that pair, and is why this list is five rather than seven.

## Cause, and why it recurs

Same divergence class as #3837, which is one of the five. There, the SQL Server migration that added `GeneratedBaseViewName` **also regenerated `vwEntities`** — its header says so explicitly — while its PostgreSQL counterpart added the column and never recreated the view. The column landed in the table and nothing surfaced it.

Nothing checks for this, so each occurrence is found only when something downstream misbehaves. For `GeneratedBaseViewName` the downstream effect was severe: `EntityInfo.GeneratedBaseViewName` was always `undefined`, `HasLayeredBaseView` was therefore always false, and `assertLayeredBaseViewSupported()` — the deliberate PostgreSQL refusal in #3477 — became unreachable (see #3837 for the full trace).

The other four have not been traced to a downstream symptom yet, which is precisely the problem: they are invisible until they are not.

## Asks

1. Expose the four remaining columns on PostgreSQL. `CREATE OR REPLACE VIEW` (append-only) rather than DROP/CREATE — PostgreSQL tracks a hard dependency from the CRUD functions to the view's row type, so a DROP needs CASCADE and takes them with it (the hazard #3477 records for `vwVersionInstallations`).
2. **Add a standing check** so this class stops being discovered by accident: every non-virtual `EntityField` whose column exists on the base table should be exposed by that entity's base view. The query above is the whole check and needs only a migrated database — the nightly PostgreSQL lane added for #3773 already has one.

## Related

- #3837 — the layered-base-view instance of this, with the full mechanism
- #3477 — the guard rendered unreachable by that instance
- #3514 — the general SS-artifact / PG-bake divergence class

Verified on PostgreSQL 17.10 with `@memberjunction/cli@6.1.0-edge.2`.

🤖 Filed with [Claude Code](https://claude.com/claude-code)

Contributor guide

Open the contributing guide

Research direction

Start by inspecting the PostgreSQL migration that added GeneratedBaseViewName and compare its view handling with the SQL Server migration described in the issue. Run the provided catalog query against a migrated PostgreSQL database, then verify that all five declared fields appear in their base views and that the standing consistency check covers this case.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.