Pinot: contains/startsWith/endsWith filters never match (LIKE against a CONCAT pattern)
- Dominant language
- Rust
- Stars
- 20.8k
- Forks
- 2.1k
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 181
Description
## Summary
On Pinot, every LIKE-based filter — `contains`, `notContains`, `startsWith`, `endsWith` — returns **no rows**, regardless of the value. This is independent of wildcard escaping and predates #11569.
It has gone unnoticed because the driver-suite snapshots record the empty results as the expected values.
## Evidence
`packages/cubejs-testing-drivers/test/__snapshots__/pinot-full.test.ts.snap` has **21** LIKE-family entries recorded as `Array []`:
```
filtering Products: contains + dimensions + order, first/second/third
contains with special chars + dimensions
startsWith …, endsWith …
filtering Customers: contains first/second/third, startsWith …, endsWith …
filtering ECommerce: contains first/second/third, startsWith …, endsWith …
```
Two of those are conclusive on their own:
- `contains 'able'` against a dataset containing **"Tables"** — no wildcard involved.
- `contains 'di_Novo'` — every other engine records the `Logitech di_Novo Edge Keyboard` row; Pinot records `Array []`.
## Cause
Cube emits, for Pinot:
```sql
LOWER(col) LIKE CONCAT('%', LOWER(?), '%')
```
Pinot does not match a **non-constant** LIKE pattern. Probed directly against `apachepinot/pinot:1.4.0` (QuickStart `baseballStats`, 97,889 rows):
```sql
LOWER(playerName) LIKE '%aaron%' -> 311
LOWER(playerName) LIKE CONCAT('%', LOWER('aaron'), '%') -> 120 -- same predicate
LOWER(playerID) LIKE CONCAT('%', LOWER('a'), '%') -> 0 -- must match
```
A literal pattern behaves correctly, including escaping:
```sql
'a_b' LIKE '%\_%' -> matches
'axb' LIKE '%\_%' -> no match
```
## Separately: the ESCAPE clause is also wrong for Pinot
`PinotFilter.likeIgnoreCase` emits `ESCAPE '\'`. Pinot **errors** on that clause:
```sql
'a_b' LIKE '%\_%' ESCAPE '\' -> Query execution error
'a_b' LIKE '%\_%' -> matches
```
Pinot already treats backslash as the escape character, so the clause should simply be removed. That fix was written and verified in #11569 and then reverted out of it, because on its own it does **not** fix the matching problem above — the driver job still failed with the snapshots unchanged (`Snapshots: 102 passed, 102 total`). It is a correct, independent change worth carrying into whatever fixes this.
## A further Pinot limitation
Even with a literal pattern, an escaped `%` immediately before a trailing wildcard is mis-parsed:
```sql
'50% off' LIKE '%\% off' -> matches
'50% off' LIKE '%\%%' -> no match -- the shape `contains` builds
```
So `contains '%'` cannot match a literal percent sign on Pinot even once the CONCAT issue is resolved.
## Suggested direction
Emit a literal LIKE pattern for Pinot rather than building it with `CONCAT`, i.e. compose the wildcards into the parameter value instead of into the SQL. That touches the `filters/like_pattern` / parameter-allocation contract for this dialect, so it needs a working Pinot to validate.
## Current state in the test suite
`packages/cubejs-testing-drivers/fixtures/pinot.json` skips one case with this reason recorded inline:
```
filtering Products: contains a literal underscore (no pre-aggregation)
```
The other four LIKE-escaping cases still run on Pinot, but they pass **only because they expect an empty result** — they are not evidence that Pinot is correct. Anything fixing this should un-skip the underscore case and expect the other four to start being meaningful.
Contributor guide
Research direction
Start with the filters/like_pattern and parameter-allocation contract, then inspect PinotFilter.likeIgnoreCase and the Pinot fixture and snapshots in packages/cubejs-testing-drivers. Run the driver suite against apachepinot/pinot:1.4.0; done means LIKE filters return matching rows, the ESCAPE issue is handled, the underscore case is no longer skipped, and snapshots reflect meaningful results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql, typescript
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100