MemberJunction / MemberJunction/MJ

CodeGen: removing a soft FK, value list or organic key from additionalSchemaInfo does not retract it from metadata

Open
#4,535 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
TSQL
Stars
29
Forks
6
Avg merge
2d 1h
Merged PRs (30d)
323

Description

## Summary

Removing a soft FK, a value list, or an organic key from `additionalSchemaInfo.json` does not remove it from MJ metadata. The file can add and change declarations, but it cannot retract them, so a correction made by deleting lines has no effect.

## Evidence

On a production deployment, a commit removed 15 self-referencing soft-FK declarations (a PK declared as an FK to itself) from the file. **14 of them were still present in `EntityField` after the next CodeGen run**, still pointing at themselves, and still generating reverse-side `EntityRelationship` rows.

The same applies to:

- **Value lists.** `manage-metadata.ts:4713-4720` skips the sync when `EntityFieldValue` rows already exist and `ValueListType` is not None, so a list deleted from the file stays published. Confirmed: a CodeGen run after removing 12 lists emitted zero `EntityFieldValue` statements.
- **Organic keys.** `processOrganicKeyConfig` only upserts `EntityOrganicKey` / `EntityOrganicKeyRelatedEntity`; nothing deactivates a key that is no longer declared. A deployment had 470 active keys on entities where the file declared 383.

On `next`, `1d9a99ec48` adds `AND NOT IsSoftForeignKey` to the schema sync, which makes soft FKs permanent: nothing clears `IsSoftForeignKey` or restores `AutoUpdateRelatedEntityInfo`, so a removed declaration can never be reverted to the physical FK either.

## Impact

The file reads as the source of truth for soft keys, but it is append/update-only in practice. Fixing bad declarations required a hand-written SQL script against `__mj` tables, which is not something a typical deployment should need.

## Suggested fix

Make the file authoritative for the entries it owns. Options:

- Track provenance (e.g. `IsSoftForeignKey` already marks these) and, on each run, clear soft keys/value lists/organic keys that are no longer declared for that entity, ideally behind a config flag so partial-file deployments are not surprised.
- Or add a documented reconciliation command (`mj codegen --reconcile-soft-keys`) that reports and optionally removes orphans.

Either way, the behaviour should be documented: today the only safe way to retract is direct SQL.

Related: #2071 (safe stale EntityRelationship cleanup), #2568 (soft composite PK leaves entity in broken state requiring manual DB fix).

---
Found alongside #4531, #4532 and #4533 during a production schema-info audit.

Contributor guide

Open the contributing guide

Research direction

Start with additionalSchemaInfo.json processing in manage-metadata.ts, especially lines 4713-4720, and trace the processOrganicKeyConfig entry point used by CodeGen. Check how soft foreign keys, value lists, and organic keys are compared with existing metadata. Done means removed declarations are reconciled without leaving stale EntityField, EntityFieldValue, EntityOrganicKey, or EntityOrganicKeyRelatedEntity records, while existing physical-FK behavior is not permanently altered.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql, typescript
Domain
backend, database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.