cube-js / cube-js/cube

Rollup pre-aggregations time ranges cache invalidation question

Open
#10,348 1 comment 0 reactions 0 assignees View on GitHub
question
Dominant language
Rust
Stars
20.8k
Forks
2.1k
Avg merge
1d 2h
Merged PRs (30d)
181

Description

**Problem**
Hi, I'm facing issues with incremental pre-aggregations run at the scheduled time (2 AM UTC), Cube frequently uses a cached build_range_end timestamp from the previous day's run instead of calculating a fresh one based on the current date/time.

Could someone explain, how exactly is build_range_end cached for incremental pre-aggregations when using Cubestore?What mechanism should trigger cache invalidation for build_range_end to ensure it's recalculated on each scheduled refresh? Thanks in advance for any help in this investigation.

### Details
**refresh key for cube rollup pre-aggregation**
```yaml
...
time_dimension: at
granularity: day
partition_granularity: month
refresh_key:
every: "0 2 * * *"
incremental: true
update_window: 6 days
...
```

Refresh worker relevant config env vars:
```
CUBEJS_TOUCH_PRE_AGG_CACHE_MAX_AGE: "3600"
CUBEJS_DROP_PRE_AGG_WITHOUT_TOUCH: "true"
CUBEJS_TOUCH_PRE_AGG_TIMEOUT: "82800"
```

**Example 1 - January 26, 2026:**
Aggregation query finished at: 2026-01-26T02:00:29
Used end period: 2026-01-25T02:05:04.000Z (from previous day from previous day aggregation)

**Example 2 - January 25, 2026:**
Aggregation query finished at: 2026-01-25T02:03:45
Used end period: 2026-01-24T02:02:30.000Z (timestamp from previous day aggregation)

Since the pre-aggregation runs daily at 2 AM UTC and CUBEJS_TOUCH_PRE_AGG_CACHE_MAX_AGE is set to 1 hour (3600 seconds) I expected the build_range_end value from yesterday's run (24 hours ago) should be expired and recalculated using default NOW() at the time of the current scheduled refresh.

Each day's scheduled refresh should use a fresh build_range_end timestamp reflecting the current date/time, not a cached value from 24 hours prior.

Attempted solutions that did NOT resolve the issue:
- Setting CUBEJS_TOUCH_PRE_AGG_CACHE_MAX_AGE to 1 hour (3600 seconds)
- Setting CUBEJS_TOUCH_PRE_AGG_TIMEOUT to 23 hours (82800 seconds)
- Various combinations of both settings above
- Setting build_range_end explicitly sql with proper period, refresh worker used value from previous day anyway but with my build_range_end sql

**Cube.js version:** 1.6.1
**Database:** ClickHouse
**Cache/Queue driver:** Cubestore
**Deployment:** Kubernetes with dedicated refresh worker

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.