MemberJunction / MemberJunction/MJ

A sync writes every row successfully against stale CRUD procedures and reports success, because nothing checks the procedure signature first

Open
#4,433 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TSQL
Stars
29
Forks
6
Avg merge
2d 1h
Merged PRs (30d)
323

Description

## What happened

A data sync ran for hours, the run counters climbed normally, and the run finished reporting success. The destination row count did not move. The cause was that the entity's generated CRUD procedures were stale relative to its current field set — the procedure took a different parameter list than the caller was passing — so every write failed at the database and the failures were counted as processed work.

**65,203 writes failed this way in a single run and the run still closed green.** The only trace was in the run's own error log, which nobody reads when a run reports success.

## Why a preflight is the right fix

The mismatch is knowable before the first row is written. At sync start we already have the entity's `EntityField` set and the database's procedure signatures. Comparing them is one query and a set difference.

Doing it once at the start turns a silent multi-hour data-loss event into a single loud refusal: *"the create procedure for this entity takes 14 parameters, the entity has 17 fields, run CodeGen."*

That is the whole ask, and it is cheap.

## Suggested shape

At sync start, for every entity the run will write to:

- read the generated CRUD procedure signatures from the catalog;
- compare against the entity's current field set;
- on a mismatch, fail the run immediately, naming the entity, the procedure, and the direction of the difference.

There is no parity or preflight check of this kind anywhere on `next` today — grepping for one finds nothing.

## The second half, stated separately

The deeper version is that CodeGen should be transactional, so a partially-applied regeneration cannot leave an entity's view and its procedures disagreeing in the first place. That is a much larger change and a separate decision; the preflight is worth doing regardless, because it also catches drift introduced by hand.

## Notes

Filed as an issue rather than a PR because the placement is yours to choose — it could live in the sync engine's startup path or in the provider — and because the transactional-CodeGen half genuinely is a design question. Happy to implement the preflight half against whatever shape you prefer.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the sync engine's startup path and the provider to find where EntityField data and database procedure signatures are available. Add a preflight for each entity being written that compares the CRUD signatures and fails with the entity, procedure, and direction of the mismatch; verify that stale procedures stop the run before any rows are written.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.