MemberJunction / MemberJunction/MJ
mj sync push writes metadata without publishing cache invalidations, so a Redis-backed server serves pre-sync data indefinitely
- Dominant language
- TSQL
- Stars
- 29
- Forks
- 6
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 323
Description
## Summary
`mj sync push` writes metadata through `BaseEntity.Save()`, which raises cache-invalidation events into **its own process's** `LocalCacheManager`. It never installs a shared storage provider, so in a deployment that uses the Redis provider for cross-server cache coherence those invalidations are never published. Every running server keeps serving the pre-push result sets.
Three things combine to make this permanent rather than transient:
1. **No invalidation is published.** `packages/MetadataSync/src/lib/provider-utils.ts` → `initializeProvider()` sets up the SQL provider only; nothing calls `SetLocalStorageProvider`. `LocalCacheManager.InvalidateEntityCaches()` fires locally and dies with the CLI process.
2. **Cached entries never expire.** `packages/MJCore/src/generic/localCacheManager.ts` ships `defaultTTLMs: 0`, so a cached `RunView` result set has no expiry.
3. **Restarting doesn't help.** Redis is external to the app, so stale entries survive redeploys and pod restarts — a fresh process reads the same stale keys.
The affected read path is the server-cache leg in `providerBase`: for an entity with `TrustServerCacheCompletely = true`, `RunView` can serve from cache with no DB hit. Entities that *also* have `TrackRecordChanges = false` have no `RecordChange` trail, so the differential / `maxUpdatedAt` freshness check isn't available either and the serve-from-cache path returns cached rows unconditionally.
Net effect: **metadata is correct in SQL and stale in every running server, with no TTL and no restart that recovers it.** The only remedy today is clearing Redis by hand.
## How we hit it
An org-scoped configuration record was corrected via `mj sync push` and verified in SQL — `__mj_UpdatedAt` moved and the stored content matched the repo. Nine days later every request was still being served the pre-correction revision, byte-identical to the superseded git commit. Two production pipelines failed outright as a direct result, because the stale record carried guidance that had already been fixed.
A redeploy in the interim did not clear it, which is what made the cause hard to see: the staleness lives in Redis, not in process memory. The subsequent metadata sync also reported `no changes` — correctly, since SQL already matched the repo — so nothing in CI indicated a problem.
This is not confined to one entity type. In our database **208 entities** carry the `TrustServerCacheCompletely = 1` + `TrackRecordChanges = 0` combination:
```sql
SELECT SchemaName, Name FROM __mj.Entity
WHERE TrustServerCacheCompletely = 1 AND TrackRecordChanges = 0
ORDER BY SchemaName, Name;
```
## Proposed fix
Have MetadataSync install the Redis provider when the environment supplies one, mirroring what `MJServer` already does in `packages/MJServer/src/index.ts` (~L616):
```ts
if (process.env.REDIS_URL) {
const redisProvider = new RedisLocalStorageProvider({
url: process.env.REDIS_URL,
keyPrefix: process.env.REDIS_KEY_PREFIX || 'mj',
enablePubSub: true,
enableLogging: false,
});
(Metadata.Provider as GenericDatabaseProvider).SetLocalStorageProvider(redisProvider);
await redisProvider.StartListening();
}
```
Wiring this into `initializeProvider()` in `provider-utils.ts` would cover every `mj sync` invocation — CI and developer laptop alike — and lets the existing per-entity invalidation flow through unchanged. Gating on `REDIS_URL` being present means no behaviour change for deployments that don't use a shared cache.
Worth considering alongside it:
- **A non-zero `defaultTTLMs`.** An entry that can never expire *and* can only be cleared by an event that never fires is unrecoverable without manual intervention. A bounded TTL turns "broken until someone notices" into "self-heals".
- **Other out-of-band writers.** The same gap plausibly applies to `mj migrate`, `mj app install` / `upgrade`, and CodeGen — they all mutate metadata from a process that isn't the API.
## Workaround
Until this lands we clear the affected cache categories from CI after every metadata push — `RunViewCache`, `RunQueryCache`, `DatasetCache` and `Metadata`, via the provider's `ClearCategory` API. `Metadata` has to go with the other three because it holds the cache's own `__MJ_CACHE_REGISTRY__`, which indexes them.
## Related
#3059 covers invalidation *ordering* within a single process. This is a distinct failure: the invalidation is never published across processes at all.
Verified against `main` @ `40c30ac6`.
Contributor guide
Research direction
Start in packages/MetadataSync/src/lib/provider-utils.ts and compare initializeProvider() with the Redis setup in packages/MJServer/src/index.ts around line 616. Review packages/MJCore/src/generic/localCacheManager.ts for the invalidation flow, then run the existing metadata sync path with REDIS_URL configured. Done means sync push publishes invalidations to Redis while deployments without REDIS_URL retain current behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- redis, typescript
- Domain
- backend, cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100