Cube generating invalid SQL through a many-to-many association
- Dominant language
- Rust
- Stars
- 20.8k
- Forks
- 2.1k
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 181
Description
**Describe the bug**
Hello! Cube is generating invalid SQL and we can't figure out why. We have users, which have phone calls. We also have teams, which have a many-to-many relationship with users, through a join table user_teams. When we query phone calls via users, everything's fine. When we query via teams however, we get invalid SQL.
This query, via users, works:
```
{"dimensions":["users.email"], "measures":["phone_calls.total_count", "phone_calls.connected_rate", "phone_calls.interested_rate"]}
```
producing the expected SQL, roughly `select from users left join phone calls`.
This query, however, via teams:
```
{"dimensions":["teams.name"], "measures":["phone_calls.total_count", "phone_calls.connected_rate"]}
```
produces a "weird" query, with two sub queries, such that the inner most sub query is a `SELECT DISTINCT`.
Things get even weirder with this query, however:
```
{"dimensions":["teams.name"], "measures":["phone_calls.total_count", "phone_calls.connected_rate", "phone_calls.interested_rate"]}
```
`phone_calls.interested_rate` requires a join from phone_calls to phone_call_dispositions. This apparently completely wrecks everything, and this time the query cube generates is invalid and produces `Error: operator does not exist: uuid = integer`. The problem is that cube "forgets" to include a `count()` statement inside the `NULLIF` in `interested_rate`, which is templated by `{total_count}`.
**To Reproduce**
Steps to reproduce the behavior:
1. Copy our cubes setup
1. Run this query: `{"dimensions":["teams.name"], "measures":["phone_calls.total_count", "phone_calls.connected_rate", "phone_calls.interested_rate"]}`
**Expected behavior**
Expecting Cube to generate valid SQL by replacing `{total_count}` correctly.
Additionally, we would expect Cube to generate a much simpler SQL query structure (see additional context).
**Minimally reproducible Cube Schema**
```yml
cubes:
- name: accounts
sql_table: public.dim__account
data_source: default
dimensions:
- name: name
sql: name
type: string
- name: account_id
sql: account_id
type: number
primary_key: true
- name: users
sql_table: public.dim__user
data_source: default
dimensions:
- name: email
sql: email
type: string
- name: user_key
sql: user_key
type: number
primary_key: true
- name: account_id
sql: account_id
type: string
joins:
- name: accounts
relationship: belongs_to
sql: "{CUBE}.account_id = {accounts.account_id}"
- name: phone_calls
relationship: has_many
sql: "{CUBE}.user_key = {phone_calls.user_key}"
- name: user_teams
sql_table: public.br__user_team
data_source: default
joins:
- name: users
relationship: belongs_to
sql: "{CUBE}.user_key = {users.user_key}"
dimensions:
- name: user_key
sql: user_key
type: string
- name: team_key
sql: team_key
type: string
- name: user_team_id
sql: user_team_id
type: number
primary_key: true
- name: account_id
sql: account_id
type: string
- name: teams
sql_table: public.dim__team
data_source: default
joins:
- name: user_teams
relationship: has_many
sql: "{CUBE}.team_key = {user_teams.team_key}"
dimensions:
- name: name
sql: name
type: string
- name: team_key
sql: team_key
type: string
primary_key: true
- name: account_id
sql: account_id
type: string
- name: phone_calls
sql_table: fact__phone_call
data_source: default
measures:
- name: total_count
type: count
sql: phone_call_id
- name: duration_sum
type: sum
sql: duration_seconds
- name: connected_count
type: number
sql: "COUNT(*) FILTER (WHERE {CUBE}.answered = TRUE AND {CUBE}.machine IS FALSE)"
- name: connected_rate
type: number
format: percent
sql: ROUND({connected_count}::decimal / NULLIF({total_count}, 0) * 100, 2)
- name: interested_count
type: number
sql: "COUNT(*) FILTER (WHERE {phone_call_dispositions.label} = 'Interested')::decimal"
- name: interested_rate
type: number
format: percent
sql: ROUND({interested_count}::decimal / NULLIF({total_count}, 0) * 100, 2)
joins:
- name: phone_call_dispositions
relationship: belongs_to
sql: "{CUBE}.phone_call_disposition_key = {phone_call_dispositions.phone_call_disposition_key}"
dimensions:
- name: phone_call_id
sql: phone_call_id
type: string
primary_key: true
- name: account_id
sql: account_id
type: string
- name: created_at
type: time
sql: created_at
- name: user_key
sql: user_key
type: string
- name: phone_call_dispositions
sql_table: public.dim__phone_call_disposition
data_source: default
dimensions:
- name: phone_call_disposition_key
sql: phone_call_disposition_key
type: string
primary_key: true
- name: label
sql: label
type: string
- name: account_id
sql: account_id
type: string
views:
- name: team_phone_calls
cubes:
- join_path: teams
includes:
- name
- join_path: teams.user_teams.users.phone_calls
includes:
- total_count
- connected_count
- connected_rate
- interested_count
- interested_rate
```
**Version:**
0.34.50
**Additional context**
Here's a detailed gist containing our cubes, queries and generated SQL: https://gist.github.com/pineman/5ed60dea8289455863841300d9fcabda
We thought using views might help control the problem as we can control join order better, and an initial query with just a simple total count measure produces the SQL we were expecting, roughly `select from teams left join user_teams left join users left join phone_calls`. But when we add just one more measure, it reverts back to the "weird" query with subqueries and `SELECT DISTINCT`. And when we add the `interested_rate` measure it fails just like above.
**Any help figuring out what we're doing wrong would be greatly appreciated! Thanks.**
Contributor guide
Assessment
This issue has not been assessed yet.