MemberJunction / MemberJunction/MJ
Cache fingerprint consistency: dataset collision + RLS mismatch in BaseEngine
- Dominant language
- TSQL
- Stars
- 29
- Forks
- 6
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 323
Description
## Context
While fixing the `BaseEngine` cache callback fingerprint mismatch in PR #3366 (missing `IgnoreMaxRows: true`), an audit of all `GenerateRunViewFingerprint` call sites revealed two additional latent consistency issues.
## Issue 1 (Medium): Dataset fingerprint collision with RunView
**Files**: `packages/GenericDatabaseProvider/src/GenericDatabaseProvider.ts` (~lines 4058, 4099, 4277)
The dataset code constructs minimal `RunViewParams` like `{ EntityName, ExtraFilter }` to generate a fingerprint. This produces the same fingerprint format as a normal `RunView` with no filter/orderby. If a RunView and a dataset query target the same entity+filter, they share a cache slot despite storing different data shapes (raw SQL columns vs full entity data).
**Potential fix**: Dataset caching could use a distinguishing prefix in the fingerprint (e.g., `ds:`) or a separate cache category entirely.
## Issue 2 (Medium-High, theoretical): RLS mismatch between engine load and invalidation
**Files**: `packages/MJCore/src/generic/baseEngine.ts` — `BuildRunViewParamsForConfig`, `syncLocalCacheForConfig`, `RegisterCacheChangeCallbacks`
When `BaseEngine` loads data via `RunView`, `ProviderBase.PreRunView` generates the fingerprint **with** `rlsWhereClause` as the third argument (producing an `rls:` suffix). But `BuildRunViewParamsForConfig` — used by `syncLocalCacheForConfig` and `RegisterCacheChangeCallbacks` — calls `GenerateRunViewFingerprint` with only two arguments (no RLS clause). If the `contextUser` has RLS configured for the target entity, the stored fingerprint has `rls:` but the invalidation/callback fingerprint does not.
**Mitigating factor**: Engine-cached entities are almost always metadata tables (AI Models, Prompts, Actions, etc.) which don't have RLS configured. This is theoretical today but becomes real the moment someone adds RLS to an engine-cached entity.
**Potential fix**: The engine would need to compute and pass the RLS where clause when generating fingerprints. `ComputeRunViewRLSWhereClause` currently lives on `ProviderBase`, so it would need to be accessible from `BaseEngine` (either via the provider reference or extracted into a shared utility).
## Related
- PR #3366 — fixed the `IgnoreMaxRows` mismatch across `LoadSingleEntityConfig`, `LoadMultipleEntityConfigs`, `RegisterCacheChangeCallbacks`, and `syncLocalCacheForConfig`
Contributor guide
Research direction
Start with the dataset fingerprint call sites in packages/GenericDatabaseProvider/src/GenericDatabaseProvider.ts, then trace BuildRunViewParamsForConfig, syncLocalCacheForConfig, and RegisterCacheChangeCallbacks in packages/MJCore/src/generic/baseEngine.ts. Compare these fingerprints with ProviderBase.PreRunView for dataset/RunView separation and RLS consistency; done means load, invalidation, and callbacks address the same cache entries without collisions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100