rollup_join regression between 1.3.26 and 1.6.69 (multi-cube, zero-measure query) — related to #10384
- Dominant language
- Rust
- Stars
- 20.8k
- Forks
- 2.1k
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 181
Description
**Describe the bug**
A multi-cube, multi-data-source `rollupJoin` pre-aggregation that matched correctly on `1.3.26` stops matching after upgrading to `1.6.69`, throwing the cross-data-source error even though a `rollupJoin` covering exactly those cubes/dimensions is defined. The query itself is unchanged between versions — only the Cube.js version changed.
This looks like the same underlying regression reported in #10384 ("rollup_join regression between 1.3 and 1.6"), which was closed with only "Need to double check if it is a real issue. closing for now" and no further investigation. Filing separately with a fuller repro since that issue was closed without resolution — please link/reopen as appropriate.
Also possibly related (different trigger conditions, but same error and same subsystem): #11124 (measure from secondary cube not matched) and #10104 (`set`/`notSet` operator not matched in `rollup_join`), though neither of those is framed as a version regression the way this and #10384 are.
**To Reproduce**
Our `rollupJoin` spans **4 cubes across 4 distinct data sources**, and the failing query selects **zero measures** (dimensions + filters only — a "list records" style query), which rules out the secondary-cube-measure trigger from #11124.
Schema (simplified, names/values anonymized):
```js
cube(`PM_Teams`, {
// dataSource: propertyManagement
preAggregations: {
glRentTenancyTeamsRollup: {
type: `rollupJoin`,
measures: [],
dimensions: [
PM_Teams.teamId, PM_Teams.propertyAddress, PM_Teams.propertyType,
Tenancies.tenancyId, Tenancies.renterNames, Tenancies.isCurrentEffectiveTenancy, Tenancies.isOngoingManagement,
GeneralLedger_RentByTenancy.rentAmount, GeneralLedger_RentByTenancy.firstRentCreatedAt, GeneralLedger_RentByTenancy.lastRentCreatedAt,
PaymentMethod.methodType,
],
rollups: [
PM_Teams.pMTeamsRollup,
Tenancies.tenancyRentersRollup, // dataSource: default
GeneralLedger_RentByTenancy.rentTenancyRollup, // dataSource: generalLedger
PaymentMethod.methodRollup, // dataSource: payment
],
},
},
});
```
Query (no measures; dimensions + filters only):
```json
{
"dimensions": [
"GeneralLedger_RentByTenancy.rentAmount",
"GeneralLedger_RentByTenancy.firstRentCreatedAt",
"GeneralLedger_RentByTenancy.lastRentCreatedAt",
"PM_Teams.propertyAddress",
"PM_Teams.propertyType",
"PM_Teams.teamId",
"PaymentMethod.methodType",
"Tenancies.renterNames",
"Tenancies.managementId"
],
"order": [["PM_Teams.propertyAddress", "asc"]],
"limit": 5000,
"filters": [
{ "or": [
{ "member": "PaymentMethod.methodType", "operator": "equals", "values": ["bank_account", "credit_card", "..."] },
{ "member": "PaymentMethod.methodType", "operator": "notSet" }
]},
{ "member": "Tenancies.isCurrentEffectiveTenancy", "operator": "equals", "values": ["true"] },
{ "member": "Tenancies.isOngoingManagement", "operator": "equals", "values": ["true"] },
{ "member": "PM_Teams.teamId", "operator": "equals", "values": ["", "...(60+ values)"] },
{ "member": "PM_Teams.organisationId", "operator": "equals", "values": [""] },
{ "member": "PM_Teams.propertyType", "operator": "equals", "values": ["Residential", "Commercial"] }
]
}
```
We bisected the query down to a minimal reproduction — removing the OR/`notSet` filter, removing `order`, and removing the time-typed dimensions (`firstRentCreatedAt`/`lastRentCreatedAt`) one at a time. Every reduction still hit the identical error, so it isn't tied to any specific filter operator or dimension type — it's structural to the 4-way cross-data-source join itself.
**Expected behavior**
Same as on `1.3.26`: the query should route through `PM_Teams.glRentTenancyTeamsRollup` since every requested dimension and filter member is declared in that `rollupJoin`'s `dimensions` list, and no measures are requested at all.
**Actual behavior**
```
To join across data sources use rollupJoin with Cube Store. If rollupJoin is defined, this error indicates it doesn't match the query. Please use Rollup Designer to verify it's definition. Found data sources: generalLedger, propertyManagement, payment, default
```
**Possible root cause**
Diffing `packages/cubejs-schema-compiler/src/adapter/PreAggregations.ts` between `v1.3.26` and `v1.6.69`, the matching function `canUsePreAggregationForTransformedQueryFn` dropped a `trimmedReferences()` normalization step that used to run every pre-aggregation's dimension/measure references through `CubeSymbols.joinHintFromPath(...).path` before comparing them against the query's own member names:
```js
// present in 1.3.26, removed in 1.6.69
// TODO remove this in favor of matching with join path
function trimmedReferences(references) {
const dimensionsTrimmed = references.dimensions.map(d => CubeSymbols.joinHintFromPath(d).path);
...
}
```
In `1.6.69` this helper and its `TODO` are gone; the matcher now compares `references.dimensions`/`references.measures` in raw form, with join-path-aware resolution only reintroduced in two narrow call sites (the `ungrouped` branch and `expandTimeDimension`) rather than universally. For a `rollupJoin` spanning many cubes/join paths (4, in our case), if any cross-cube reference ends up carrying a join-hint-qualified path internally, the removed trimming step would previously have normalized it back to a plain `Cube.member` string before comparison — without it, a legitimate match can silently fail on string comparison. This would explain why simpler, same-data-source rollups are unaffected while multi-cube/multi-data-source `rollupJoin`s regress, and it's consistent with both this report and #10384 (2 cubes/2 data sources) failing the same way despite differently-shaped schemas.
We haven't confirmed this against a debugger/live instance yet — flagging as a starting point, not a confirmed diagnosis.
**Version**
Regressed between `1.3.26` and `1.6.69`. Confirmed still present on `1.6.69` (latest LTS at time of writing).
Contributor guide
Research direction
Start in packages/cubejs-schema-compiler/src/adapter/PreAggregations.ts at canUsePreAggregationForTransformedQueryFn, and compare the matching logic between v1.3.26 and v1.6.69, especially trimmedReferences() and join-path handling. Reproduce the minimal four-cube, four-data-source rollupJoin with the zero-measure query; done means the query matches PM_Teams.glRentTenancyTeamsRollup as it did on 1.3.26.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100