apache / apache/superset

Boolean filters does not work for Databricks datasets

Open
#36,765 5 comments 2 reactions 0 assignees View on GitHub
data:connect:databricks 🦾 ai-candidate
Dominant language
Python
Stars
74.8k
Forks
18.3k
Avg merge
2d 4h
Merged PRs (30d)
664

Description

### Bug description

Creating a native filter with a boolean column causes the Databricks query to fail, likely due to an SQL translation issue.

Here is the filter we add.

Image

And this is the error whenever we try to set any value on this filter

```
Error: [DATATYPE_MISMATCH.DATA_DIFF_TYPES] Cannot resolve "(is_test_user IN (0))" due to data type mismatch: Input to `in` should all be the same type, but it's ["BOOLEAN", "INT"]. SQLSTATE: 42K09; line 6 pos 15

```

This is the query superset sending to databricks.

```
SELECT
...
FROM prod.video_coin_operation_extended
WHERE
is_test_user IN (0)
...
```

I think query should have sent is as `False` rather then 0
Tested it on master and version 6.0.0

We worked around this issue by creating a string-type calculated column for filtering but would still appreciate a resolution.

### Screenshots/recordings

_No response_

### Superset version

master / latest-dev

### Python version

3.9

### Node version

16

### Browser

Chrome

### Additional context

_No response_

### Checklist

- [ ] I have searched Superset docs and Slack and didn't find a solution to my problem.
- [ ] I have searched the GitHub issue tracker and didn't find a similar bug report.
- [ ] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the native boolean filter against a Databricks dataset and trace the query translation that produces `is_test_user IN (0)`. Done means the generated query uses a boolean value such as `False` and the filter succeeds without the string-column workaround.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, sql
Domain
backend, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.