cube-js / cube-js/cube

What is the correct setup for API instances VS worker instance in regards to CUBEJS_REFRESH_WORKER

Open
#9,892 3 comments 0 reactions 1 assignee Claimed by @igorlukanin View on GitHub
pre-aggregations question
Dominant language
Rust
Stars
20.8k
Forks
2.1k
Avg merge
1d 2h
Merged PRs (30d)
181

Description

**Problem**

Our cube instance setup is following the[ production checklist](https://cube.dev/docs/product/deployment/production-checklist#disable-development-moder):

1. We have 2 API instances with:
```
CUBEJS_DEV_MODE=false (explicit setup)
CUBEJS_REFRESH_WORKER = false (default)
CUBEJS_ROLLUP_ONLY = false (default)
```
2. We have 1 worker instance with:
```
CUBEJS_DEV_MODE=false (explicit setup)
CUBEJS_REFRESH_WORKER = true (explicit setup)
CUBEJS_ROLLUP_ONLY = false (default)
```
followed the recommendation [here](https://cube.dev/docs/product/deployment/production-checklist#set-up-refresh-worker)

But this setup is causing issue of
```
Error: No pre-aggregation partitions were built yet for the pre-aggregation serving this query and this API instance wasn't set up to build pre-aggregations. Please make sure your refresh worker is configured correctly, running, pre-aggregation tables are built and all pre-aggregation refresh settings like timezone match.
```
when we querying the related cube. However, what [we expect](https://cube.dev/docs/product/caching/using-pre-aggregations#matching-queries) is:

>When executing a query, Cube will try to [match and fulfill it with a pre-aggregation](https://cube.dev/docs/product/caching/matching-pre-aggregations) in the first place. If there's no matching pre-aggregation, Cube will query the upstream data source instead, unless the [rollup-only mode](https://cube.dev/docs/product/caching/using-pre-aggregations#rollup-only-mode) is enabled.

As rollup only mode is false in the both type of instances, but it looks like the API instance is behaving like `CUBEJS_ROLLUP_ONLY=true`

I also noticed[ preAggregationsOptions.externalRefresh](https://cube.dev/docs/product/configuration/reference/config#orchestrator_options), we did not explicit set it up, so by default it is `false`
> When running a separate instance of Cube to refresh pre-aggregations in the background, this option can be set on the API instance to prevent it from trying to check for rollup data being current - it won't try to create or refresh them when this option is true

But the API instance behaves like it is `true`, as it didn't try to create or refresh them.

Another thing I don’t know is how to toggle externalRefresh using ENV VARs, it doesn’t seem to have this ENV VAR, only available in [orchestrator_options](https://cube.dev/docs/product/configuration/reference/config#orchestrator_options)

Other cubejs related env (API instance):
```
CUBEJS_CACHE_AND_QUEUE_DRIVER=cubestore
CUBEJS_DB_TYPE=redshift
CUBEJS_DB_NAME=dev
CUBEJS_DB_PORT=5439
CUBEJS_DEV_MODE=false
```

Worker:
···
CUBEJS_CACHE_AND_QUEUE_DRIVER=cubestore
CUBEJS_DB_TYPE=redshift
CUBEJS_DB_NAME=dev
CUBEJS_DB_PORT=5439
CUBEJS_REFRESH_WORKER=true
CUBEJS_DEV_MODE=false
···

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.