cube-js / cube-js/cube

Customize which queries are refreshed

Open
#4,650 2 comments 0 reactions 1 assignee Claimed by @hassankhan View on GitHub
docs reviewed
Dominant language
Rust
Stars
20.8k
Forks
2.1k
Avg merge
1d 2h
Merged PRs (30d)
181

Description

We are using CubeJS based analytics with a number of users, and as of right now, Redis stores about 1000 saved query results.
Some of these queries are very common - e.g. total orders per day - I would like them to auto-refresh. Some of them are very special - the number of orders delivered to the North Pole via albatross delivery. Nobody will run it again.

Right now, I have two options:
- Don't run a separate refresh worker, and CubeJS will refresh the queries on-demand, at the cost of latency
- Run a separate refresh worker, which, per documentation, will be refreshing all saved queries - including the queries nobody needs.

To be honest, I haven't yet tried - for fear it will be actually bring the database down and/or incur BQ costs.

I would like a way to refresh only query results that were used in the last hour, or day.

I looked whether Redis TTL can be used here, but what I see if that
- Redis TTL is either specified in the query or defaults to 24 hours, I can't specify a different default via config
- If Redis TTL is set to 24 hours, then refresh worker will refresh all saved queries each 1 hour, and potentially reset the TTL, not solving my problem at all.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.