MemberJunction / MemberJunction/MJ
Materialized RunQuery: cache the per-read MR status lookup + memoize the order-by parse
- Dominant language
- TSQL
- Stars
- 29
- Forks
- 6
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 323
Description
Follow-up from the Query & Entity Materialization PR (#3735), split out so it survives PR-body deletion on merge.
**Observation (rkihm-BC re-review):** every materialized `RunQuery(DataSource:'Materialized')` read currently pays three uncached metadata operations before a single snapshot row is read:
- `tryBuildMaterializedQueryPlan` does **two** `RunView`s — the `MJ: Materialized Result Queries` join lookup, then the `MJ: Materialized Results` row read with `BypassCache: true`.
- plus an un-memoized `SQLParser.Astify` (`queryHasTopLevelOrderBy`) of the platform SQL.
For a feature whose purpose is making large reads cheap, three uncached metadata ops per read is a real tax, and on small/medium result sets it could be a net loss.
**Constraint:** the `BypassCache` on the status read is deliberate and correct — `DriftHold`/`Disabled` are written out of band by CodeGen via direct SQL (no BaseEntity cache-invalidation event), so a cached `Active` status would keep serving a held/disabled snapshot (the exact stale-serve the DriftHold mechanism prevents). Any cache here must not defeat that.
**Proposed fix:**
- A short-TTL cache keyed on MaterializedResult ID that **any status write invalidates**, so the freshness guarantee is preserved while eliminating the repeated point lookup.
- Memoize `queryHasTopLevelOrderBy` per (query SQL + platform).
Deferred from the PR intentionally (correctness-sensitive vs. the DriftHold freshness guarantee); should be its own focused change.
Contributor guide
Research direction
Start at tryBuildMaterializedQueryPlan and the queryHasTopLevelOrderBy path described in the issue, then review PR #3735 for surrounding materialized-read behavior. Verify the MaterializedResult status lookup and query SQL/platform parse paths, including the DriftHold and Disabled freshness constraint. Done means repeated materialized reads avoid the redundant metadata and parse work without serving an out-of-date status.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql
- Domain
- databases, performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100