MemberJunction / MemberJunction/MJ
CodeGen: removing a soft FK, value list or organic key from additionalSchemaInfo does not retract it from metadata
- Dominant language
- TSQL
- Stars
- 29
- Forks
- 6
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 323
Description
## Summary
Removing a soft FK, a value list, or an organic key from `additionalSchemaInfo.json` does not remove it from MJ metadata. The file can add and change declarations, but it cannot retract them, so a correction made by deleting lines has no effect.
## Evidence
On a production deployment, a commit removed 15 self-referencing soft-FK declarations (a PK declared as an FK to itself) from the file. **14 of them were still present in `EntityField` after the next CodeGen run**, still pointing at themselves, and still generating reverse-side `EntityRelationship` rows.
The same applies to:
- **Value lists.** `manage-metadata.ts:4713-4720` skips the sync when `EntityFieldValue` rows already exist and `ValueListType` is not None, so a list deleted from the file stays published. Confirmed: a CodeGen run after removing 12 lists emitted zero `EntityFieldValue` statements.
- **Organic keys.** `processOrganicKeyConfig` only upserts `EntityOrganicKey` / `EntityOrganicKeyRelatedEntity`; nothing deactivates a key that is no longer declared. A deployment had 470 active keys on entities where the file declared 383.
On `next`, `1d9a99ec48` adds `AND NOT IsSoftForeignKey` to the schema sync, which makes soft FKs permanent: nothing clears `IsSoftForeignKey` or restores `AutoUpdateRelatedEntityInfo`, so a removed declaration can never be reverted to the physical FK either.
## Impact
The file reads as the source of truth for soft keys, but it is append/update-only in practice. Fixing bad declarations required a hand-written SQL script against `__mj` tables, which is not something a typical deployment should need.
## Suggested fix
Make the file authoritative for the entries it owns. Options:
- Track provenance (e.g. `IsSoftForeignKey` already marks these) and, on each run, clear soft keys/value lists/organic keys that are no longer declared for that entity, ideally behind a config flag so partial-file deployments are not surprised.
- Or add a documented reconciliation command (`mj codegen --reconcile-soft-keys`) that reports and optionally removes orphans.
Either way, the behaviour should be documented: today the only safe way to retract is direct SQL.
Related: #2071 (safe stale EntityRelationship cleanup), #2568 (soft composite PK leaves entity in broken state requiring manual DB fix).
---
Found alongside #4531, #4532 and #4533 during a production schema-info audit.
Contributor guide
Research direction
Start with additionalSchemaInfo.json processing in manage-metadata.ts, especially lines 4713-4720, and trace the processOrganicKeyConfig entry point used by CodeGen. Check how soft foreign keys, value lists, and organic keys are compared with existing metadata. Done means removed declarations are reconciled without leaving stale EntityField, EntityFieldValue, EntityOrganicKey, or EntityOrganicKeyRelatedEntity records, while existing physical-FK behavior is not permanently altered.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql, typescript
- Domain
- backend, database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100