Customize which queries are refreshed
- Dominant language
- Rust
- Stars
- 20.8k
- Forks
- 2.1k
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 181
Description
We are using CubeJS based analytics with a number of users, and as of right now, Redis stores about 1000 saved query results.
Some of these queries are very common - e.g. total orders per day - I would like them to auto-refresh. Some of them are very special - the number of orders delivered to the North Pole via albatross delivery. Nobody will run it again.
Right now, I have two options:
- Don't run a separate refresh worker, and CubeJS will refresh the queries on-demand, at the cost of latency
- Run a separate refresh worker, which, per documentation, will be refreshing all saved queries - including the queries nobody needs.
To be honest, I haven't yet tried - for fear it will be actually bring the database down and/or incur BQ costs.
I would like a way to refresh only query results that were used in the last hour, or day.
I looked whether Redis TTL can be used here, but what I see if that
- Redis TTL is either specified in the query or defaults to 24 hours, I can't specify a different default via config
- If Redis TTL is set to 24 hours, then refresh worker will refresh all saved queries each 1 hour, and potentially reset the TTL, not solving my problem at all.
Contributor guide
Assessment
This issue has not been assessed yet.