apache / apache/hudi

read_blob() in filter predicate fails with INTERNAL_ERROR instead of clean analyzer rejection

Open
#18,820 1 comment 0 reactions 0 assignees View on GitHub
type:bug
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

**What happened:**
Using `read_blob()` inside a WHERE predicate fails with:

```
[INTERNAL_ERROR] Cannot generate code for expression: read_blob(...)
```

Example query:
```sql
SELECT id FROM t WHERE length(read_blob(image_bytes)) = 11;
```

`read_blob()` works correctly in the SELECT list — only filter predicates trigger the codegen failure.

**What you expected:**
Two things:
1. The codegen restriction should surface as an analyzer-level rejection with a clear "read_blob() is not supported in filter predicates" message, not an INTERNAL_ERROR with a Spark codegen stack trace.
2. Docs (AI quick start) should call out the recommended workaround: for length-based filtering, filter on the BLOB struct's `.length` subfield from the meta columns (e.g. `WHERE image_bytes.length = 11`) rather than wrapping `read_blob()` in `length(...)`. Typical usage is vector search or filtering on structured columns; pulling raw bytes through codegen in a predicate is not a supported path.

**Steps to reproduce:**
1. Use 1.2.0-rc2 Spark bundle.
2. Create a table with a BLOB column `image_bytes` and insert rows.
3. Run: `SELECT id FROM t WHERE length(read_blob(image_bytes)) = 11`.
4. Observe INTERNAL_ERROR.

**Environment:**
- Hudi version: 1.2.0-rc2
- Query engine: Spark 3.5
- Found during: 1.2.0-rc2

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the SQL example on the Spark 3.5 bundle, then trace how read_blob() is handled when it appears inside a WHERE predicate versus the SELECT list. Update the rejection so this unsupported use produces a clear analyzer error, and update the AI quick start with the .length workaround; verify that the query no longer fails with INTERNAL_ERROR.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
databases, documentation
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.