cube-js / cube-js/cube

Pinot: contains/startsWith/endsWith filters never match (LIKE against a CONCAT pattern)

Open
#11,570 2 comments 0 reactions 0 assignees View on GitHub
data source driver driver:pinot
Dominant language
Rust
Stars
20.8k
Forks
2.1k
Avg merge
1d 2h
Merged PRs (30d)
181

Description

## Summary

On Pinot, every LIKE-based filter — `contains`, `notContains`, `startsWith`, `endsWith` — returns **no rows**, regardless of the value. This is independent of wildcard escaping and predates #11569.

It has gone unnoticed because the driver-suite snapshots record the empty results as the expected values.

## Evidence

`packages/cubejs-testing-drivers/test/__snapshots__/pinot-full.test.ts.snap` has **21** LIKE-family entries recorded as `Array []`:

```
filtering Products: contains + dimensions + order, first/second/third
contains with special chars + dimensions
startsWith …, endsWith …
filtering Customers: contains first/second/third, startsWith …, endsWith …
filtering ECommerce: contains first/second/third, startsWith …, endsWith …
```

Two of those are conclusive on their own:

- `contains 'able'` against a dataset containing **"Tables"** — no wildcard involved.
- `contains 'di_Novo'` — every other engine records the `Logitech di_Novo Edge Keyboard` row; Pinot records `Array []`.

## Cause

Cube emits, for Pinot:

```sql
LOWER(col) LIKE CONCAT('%', LOWER(?), '%')
```

Pinot does not match a **non-constant** LIKE pattern. Probed directly against `apachepinot/pinot:1.4.0` (QuickStart `baseballStats`, 97,889 rows):

```sql
LOWER(playerName) LIKE '%aaron%' -> 311
LOWER(playerName) LIKE CONCAT('%', LOWER('aaron'), '%') -> 120 -- same predicate
LOWER(playerID) LIKE CONCAT('%', LOWER('a'), '%') -> 0 -- must match
```

A literal pattern behaves correctly, including escaping:

```sql
'a_b' LIKE '%\_%' -> matches
'axb' LIKE '%\_%' -> no match
```

## Separately: the ESCAPE clause is also wrong for Pinot

`PinotFilter.likeIgnoreCase` emits `ESCAPE '\'`. Pinot **errors** on that clause:

```sql
'a_b' LIKE '%\_%' ESCAPE '\' -> Query execution error
'a_b' LIKE '%\_%' -> matches
```

Pinot already treats backslash as the escape character, so the clause should simply be removed. That fix was written and verified in #11569 and then reverted out of it, because on its own it does **not** fix the matching problem above — the driver job still failed with the snapshots unchanged (`Snapshots: 102 passed, 102 total`). It is a correct, independent change worth carrying into whatever fixes this.

## A further Pinot limitation

Even with a literal pattern, an escaped `%` immediately before a trailing wildcard is mis-parsed:

```sql
'50% off' LIKE '%\% off' -> matches
'50% off' LIKE '%\%%' -> no match -- the shape `contains` builds
```

So `contains '%'` cannot match a literal percent sign on Pinot even once the CONCAT issue is resolved.

## Suggested direction

Emit a literal LIKE pattern for Pinot rather than building it with `CONCAT`, i.e. compose the wildcards into the parameter value instead of into the SQL. That touches the `filters/like_pattern` / parameter-allocation contract for this dialect, so it needs a working Pinot to validate.

## Current state in the test suite

`packages/cubejs-testing-drivers/fixtures/pinot.json` skips one case with this reason recorded inline:

```
filtering Products: contains a literal underscore (no pre-aggregation)
```

The other four LIKE-escaping cases still run on Pinot, but they pass **only because they expect an empty result** — they are not evidence that Pinot is correct. Anything fixing this should un-skip the underscore case and expect the other four to start being meaningful.

Contributor guide

Open the contributing guide

Research direction

Start with the filters/like_pattern and parameter-allocation contract, then inspect PinotFilter.likeIgnoreCase and the Pinot fixture and snapshots in packages/cubejs-testing-drivers. Run the driver suite against apachepinot/pinot:1.4.0; done means LIKE filters return matching rows, the ESCAPE issue is handled, the underscore case is no longer skipped, and snapshots reflect meaningful results.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql, typescript
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.