MemberJunction / MemberJunction/MJ
A sync writes every row successfully against stale CRUD procedures and reports success, because nothing checks the procedure signature first
- Dominant language
- TSQL
- Stars
- 29
- Forks
- 6
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 323
Description
## What happened
A data sync ran for hours, the run counters climbed normally, and the run finished reporting success. The destination row count did not move. The cause was that the entity's generated CRUD procedures were stale relative to its current field set — the procedure took a different parameter list than the caller was passing — so every write failed at the database and the failures were counted as processed work.
**65,203 writes failed this way in a single run and the run still closed green.** The only trace was in the run's own error log, which nobody reads when a run reports success.
## Why a preflight is the right fix
The mismatch is knowable before the first row is written. At sync start we already have the entity's `EntityField` set and the database's procedure signatures. Comparing them is one query and a set difference.
Doing it once at the start turns a silent multi-hour data-loss event into a single loud refusal: *"the create procedure for this entity takes 14 parameters, the entity has 17 fields, run CodeGen."*
That is the whole ask, and it is cheap.
## Suggested shape
At sync start, for every entity the run will write to:
- read the generated CRUD procedure signatures from the catalog;
- compare against the entity's current field set;
- on a mismatch, fail the run immediately, naming the entity, the procedure, and the direction of the difference.
There is no parity or preflight check of this kind anywhere on `next` today — grepping for one finds nothing.
## The second half, stated separately
The deeper version is that CodeGen should be transactional, so a partially-applied regeneration cannot leave an entity's view and its procedures disagreeing in the first place. That is a much larger change and a separate decision; the preflight is worth doing regardless, because it also catches drift introduced by hand.
## Notes
Filed as an issue rather than a PR because the placement is yours to choose — it could live in the sync engine's startup path or in the provider — and because the transactional-CodeGen half genuinely is a design question. Happy to implement the preflight half against whatever shape you prefer.
Contributor guide
Research direction
Start by tracing the sync engine's startup path and the provider to find where EntityField data and database procedure signatures are available. Add a preflight for each entity being written that compares the CRUD signatures and fails with the entity, procedure, and direction of the mismatch; verify that stale procedures stop the run before any rows are written.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100