MemberJunction / MemberJunction/MJ
Second-order blockers detected by the Phase-0 retirement pre-clean have no supported cleanup path
- Dominant language
- TSQL
- Stars
- 29
- Forks
- 6
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 323
Description
**Found by** the PR gauntlet on #4489 (`fix/4483-retirement-preclean`), 2026-09-14. Requested by the
reviewer on that PR as the agreed follow-up. Not a defect in #4489 — that PR deliberately *detects*
these chains and refuses, rather than deleting user data unattended. This issue is the opt-in
cleanup path that refusal implies.
### Where
`migrations/v6/V202608061704__v6.1.x__Phase0_Legacy_Workflow_Report_ScheduledAction_Retirement.sql`
— the "Second-order blockers" block (the `#ProcScoped` / `#SecondOrder` / `#Blocked` machinery).
Related: `packages/CodeGenLib/src/Database/manage-metadata.ts:4147`, the only runtime caller of
`spDeleteEntityWithCoreDependencies`.
### What happens
#4489's pre-clean clears every **first-order** inbound foreign key into `__mj.Entity` for the 11
retired Workflow / Report / Scheduled Action entities, so the retirement migration now succeeds on
databases where those features were used.
It cannot clear **second-order** blockers: rows that reference a row the cascade itself is about to
delete. Those tables have no foreign key into `Entity` at all, so no catalog sweep rooted at
`Entity` can discover them. #4489 detects them and aborts with an actionable message before
modifying anything:
```
Retirement pre-clean: rows reference records that the entity-deletion cascade is about to delete
for the retired entities. They would fail it mid-way and leave the metadata half-pruned, so the
migration stops here instead. Blocking (referencing table.column -> table being cleared):
__mj.ConversationDetail.ConversationID -> Conversation (1 row). Clear these rows, then re-run the
migration; this migration will not delete them for you -- they are user data.
```
The known chains, as documented in #4489:
| the cascade deletes | still-blocking dependents |
|---|---|
| `Conversation` (`LinkedEntityID`) | `ConversationDetail`, `ConversationArtifact`, `AIAgentSession`, `UserRoutine` |
| `EntityDocument` | `EntityDocumentRun`, `EntityDocumentSetting`, `EntityRecordDocument`, `ContentSource` |
| `List` | `DuplicateRun.SourceListID`, `RecordProcess.ScopeListID` |
### Why it matters
A database holding any of these rows **cannot upgrade unattended**. The operator gets a clear
message naming the table and row count, but no supported way to act on it other than hand-writing
`DELETE` statements against conversation history, agent sessions and vectorization state — exactly
the data nobody wants deleted by improvisation at 2am during an upgrade window. Today the only
options are "write the DELETEs yourself" or "stay on 5.51.2". There is no opt-in switch, no preview
of what would be removed, and no record of what was removed.
### Repro
```bash
# A stock 5.51.2 database, built from the released migration slice:
mj migrate --tag v5.51.2 # DB_DATABASE pointed at a scratch database
# Give it one ordinary second-order row:
# INSERT INTO __mj.Conversation (..., LinkedEntityID, LinkedRecordID, ...)
# VALUES (..., '09248F34-2837-EF11-86D4-6045BDEE16E6', '', ...) -- MJ: Reports
# INSERT INTO __mj.ConversationDetail (..., ConversationID, ...) VALUES (..., , ...)
mj migrate --dir ./migrations # upgrade to 6.1.x
# -> fails in V202608061704's pre-clean with the message above, at batch 20/34,
# before anything is modified.
```
### Evidence
Gauntlet row **SR9** on #4489. The abort is clean and replayable, which is what makes an opt-in
path safe to build on:
```
history row for 202608061704 0 (no history row written)
any failed history row 0 (rolled back completely)
retired entities still present 11
EntityField rows for them 129 (NOT stripped)
EntityPermission rows for them 28 (NOT stripped)
the blocking ConversationDetail 1 (user data untouched)
hwm 202608052200 (the migration BEFORE 061704)
```
And row **SR9-rerun**: after deleting only the rows the message named, `mj migrate` replays cleanly
— 57 applied, 0 failures, all 11 entities retired. So the operator-side loop already works; what is
missing is a supported way to perform that deletion.
### Suggested fix
An **opt-in** cleanup, off by default. Sketch, in preference order:
1. A separate, explicitly-invoked migration or `mj` command (e.g. `mj migrate preclean-retired
--confirm`) that clears the three chains above. It must not run as part of an ordinary
`mj migrate`.
2. A dry-run mode that prints exactly what would be deleted, per table, with row counts — reusing
the `#Blocked` query #4489 already builds, so the preview and the deletion cannot drift.
3. A record of what was removed, written before deletion.
It belongs outside `spDeleteEntityWithCoreDependencies`. That proc is generic metadata plumbing
called by CodeGen on every run; teaching it to delete conversation history would make every future
CodeGen run capable of destroying user data as a side effect. #4489's own header states this
reasoning and it should be preserved.
### Definition of done
- [ ] A failing test that reproduces a second-order block, then green after the opt-in path runs
- [ ] An ordinary `mj migrate` with no flag still **refuses** and still names the blocking rows —
the default behaviour from #4489 is unchanged
- [ ] The dry-run output and the actual deletion are generated from one query, with a test that
would fail if they diverged
- [ ] Existing suite and gates green; no changed expectation in an existing test
### Verify by
Build the repro above, run the opt-in command in dry-run and confirm it names
`__mj.ConversationDetail.ConversationID -> Conversation (1 row)`; run it for real; then
`mj migrate --dir ./migrations` completes and all 11 retired entities are gone
(`SELECT COUNT(*) FROM __mj.Entity WHERE ID IN ()` returns 0).
Refs #4489, #3546, #4483.
Contributor guide
Assessment
This issue has not been assessed yet.