MemberJunction / MemberJunction/MJ

mj sync push writes metadata without publishing cache invalidations, so a Redis-backed server serves pre-sync data indefinitely

Open
#4,083 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
TSQL
Stars
29
Forks
6
Avg merge
2d 1h
Merged PRs (30d)
323

Description

## Summary

`mj sync push` writes metadata through `BaseEntity.Save()`, which raises cache-invalidation events into **its own process's** `LocalCacheManager`. It never installs a shared storage provider, so in a deployment that uses the Redis provider for cross-server cache coherence those invalidations are never published. Every running server keeps serving the pre-push result sets.

Three things combine to make this permanent rather than transient:

1. **No invalidation is published.** `packages/MetadataSync/src/lib/provider-utils.ts` → `initializeProvider()` sets up the SQL provider only; nothing calls `SetLocalStorageProvider`. `LocalCacheManager.InvalidateEntityCaches()` fires locally and dies with the CLI process.
2. **Cached entries never expire.** `packages/MJCore/src/generic/localCacheManager.ts` ships `defaultTTLMs: 0`, so a cached `RunView` result set has no expiry.
3. **Restarting doesn't help.** Redis is external to the app, so stale entries survive redeploys and pod restarts — a fresh process reads the same stale keys.

The affected read path is the server-cache leg in `providerBase`: for an entity with `TrustServerCacheCompletely = true`, `RunView` can serve from cache with no DB hit. Entities that *also* have `TrackRecordChanges = false` have no `RecordChange` trail, so the differential / `maxUpdatedAt` freshness check isn't available either and the serve-from-cache path returns cached rows unconditionally.

Net effect: **metadata is correct in SQL and stale in every running server, with no TTL and no restart that recovers it.** The only remedy today is clearing Redis by hand.

## How we hit it

An org-scoped configuration record was corrected via `mj sync push` and verified in SQL — `__mj_UpdatedAt` moved and the stored content matched the repo. Nine days later every request was still being served the pre-correction revision, byte-identical to the superseded git commit. Two production pipelines failed outright as a direct result, because the stale record carried guidance that had already been fixed.

A redeploy in the interim did not clear it, which is what made the cause hard to see: the staleness lives in Redis, not in process memory. The subsequent metadata sync also reported `no changes` — correctly, since SQL already matched the repo — so nothing in CI indicated a problem.

This is not confined to one entity type. In our database **208 entities** carry the `TrustServerCacheCompletely = 1` + `TrackRecordChanges = 0` combination:

```sql
SELECT SchemaName, Name FROM __mj.Entity
WHERE TrustServerCacheCompletely = 1 AND TrackRecordChanges = 0
ORDER BY SchemaName, Name;
```

## Proposed fix

Have MetadataSync install the Redis provider when the environment supplies one, mirroring what `MJServer` already does in `packages/MJServer/src/index.ts` (~L616):

```ts
if (process.env.REDIS_URL) {
const redisProvider = new RedisLocalStorageProvider({
url: process.env.REDIS_URL,
keyPrefix: process.env.REDIS_KEY_PREFIX || 'mj',
enablePubSub: true,
enableLogging: false,
});
(Metadata.Provider as GenericDatabaseProvider).SetLocalStorageProvider(redisProvider);
await redisProvider.StartListening();
}
```

Wiring this into `initializeProvider()` in `provider-utils.ts` would cover every `mj sync` invocation — CI and developer laptop alike — and lets the existing per-entity invalidation flow through unchanged. Gating on `REDIS_URL` being present means no behaviour change for deployments that don't use a shared cache.

Worth considering alongside it:

- **A non-zero `defaultTTLMs`.** An entry that can never expire *and* can only be cleared by an event that never fires is unrecoverable without manual intervention. A bounded TTL turns "broken until someone notices" into "self-heals".
- **Other out-of-band writers.** The same gap plausibly applies to `mj migrate`, `mj app install` / `upgrade`, and CodeGen — they all mutate metadata from a process that isn't the API.

## Workaround

Until this lands we clear the affected cache categories from CI after every metadata push — `RunViewCache`, `RunQueryCache`, `DatasetCache` and `Metadata`, via the provider's `ClearCategory` API. `Metadata` has to go with the other three because it holds the cache's own `__MJ_CACHE_REGISTRY__`, which indexes them.

## Related

#3059 covers invalidation *ordering* within a single process. This is a distinct failure: the invalidation is never published across processes at all.

Verified against `main` @ `40c30ac6`.

Contributor guide

Open the contributing guide

Research direction

Start in packages/MetadataSync/src/lib/provider-utils.ts and compare initializeProvider() with the Redis setup in packages/MJServer/src/index.ts around line 616. Review packages/MJCore/src/generic/localCacheManager.ts for the invalidation flow, then run the existing metadata sync path with REDIS_URL configured. Done means sync push publishes invalidations to Redis while deployments without REDIS_URL retain current behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
redis, typescript
Domain
backend, cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.