MemberJunction / MemberJunction/MJ
mj sync push silently skips records whose @file: referenced content changed (checksum covers JSON only)
- Dominant language
- TSQL
- Stars
- 29
- Forks
- 6
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 323
Description
## Summary
`mj sync push` decides whether a metadata record changed by comparing the `sync.checksum` stored in its `.json` file. That checksum is computed from the **JSON only** — it does not cover the contents of files pulled in by `@file:` references. So editing a `@file:`-referenced file and pushing is a **silent no-op**: the record is reported as unchanged and skipped, while the file on disk and the value in the database now disagree.
## Repro
1. Create a query metadata record whose SQL lives in a separate file:
```json
[
{
"fields": {
"Name": "Housing Occupancy",
"SQL": "@file:SQL/housing-occupancy.sql",
"Status": "Approved"
},
"primaryKey": { "ID": "..." }
}
]
```
2. `mj sync push --dir metadata` — the record is created and a `sync` block with a checksum is written back into the `.json`.
3. **Edit only `SQL/housing-occupancy.sql`** — any change; in our case replacing a schema reference.
4. `mj sync push --dir metadata` again.
**Expected:** the record is updated, because its effective content changed.
**Actual:** the directory is reported as unchanged and the record is skipped. The database keeps the old SQL. Nothing warns that a referenced file is newer than the checksum.
Deleting the `sync` block from the `.json` (or otherwise editing the JSON) forces the push through, which confirms the checksum is the deciding factor.
## Why it is easy to hit
`@file:` exists precisely so that large values — SQL, TypeScript interfaces, templates, prompts — live in their own files rather than being embedded in JSON. That means **the interesting content is exactly the content the checksum does not cover.** A round of "edit the SQL, push, test" produces no change and no message, and the natural conclusion is that the SQL is wrong rather than that it was never pushed.
It is also silent in the direction that matters: had it errored, or warned "referenced file is newer than checksum", it would cost a developer seconds instead of a debugging session.
## Suggested fix
Fold referenced file content into the checksum — resolve every `@file:` when computing it, so the hash covers the record's effective value rather than its literal JSON. That keeps the existing skip-unchanged optimisation intact while making it correct.
A cheaper interim option: compare the mtime of each `@file:` target against the `sync.lastModified` already stored alongside the checksum, and treat "referenced file newer" as changed (or at minimum warn).
## Environment
`@memberjunction/cli` 6.1.0-edge.4, SQL Server, macOS. Reproduced while authoring application and query metadata for a course built on MJ.
Contributor guide
Research direction
Start at the @memberjunction/cli entry point for `mj sync push` and trace how the `sync.checksum` in each `.json` file is compared. Reproduce the issue with a `@file:SQL/housing-occupancy.sql` reference, then verify that changing the referenced file causes the record to update instead of being skipped while unchanged records still use the optimization.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql
- Domain
- cli, database
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 70/100