read_blob() in filter predicate fails with INTERNAL_ERROR instead of clean analyzer rejection
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
**What happened:**
Using `read_blob()` inside a WHERE predicate fails with:
```
[INTERNAL_ERROR] Cannot generate code for expression: read_blob(...)
```
Example query:
```sql
SELECT id FROM t WHERE length(read_blob(image_bytes)) = 11;
```
`read_blob()` works correctly in the SELECT list — only filter predicates trigger the codegen failure.
**What you expected:**
Two things:
1. The codegen restriction should surface as an analyzer-level rejection with a clear "read_blob() is not supported in filter predicates" message, not an INTERNAL_ERROR with a Spark codegen stack trace.
2. Docs (AI quick start) should call out the recommended workaround: for length-based filtering, filter on the BLOB struct's `.length` subfield from the meta columns (e.g. `WHERE image_bytes.length = 11`) rather than wrapping `read_blob()` in `length(...)`. Typical usage is vector search or filtering on structured columns; pulling raw bytes through codegen in a predicate is not a supported path.
**Steps to reproduce:**
1. Use 1.2.0-rc2 Spark bundle.
2. Create a table with a BLOB column `image_bytes` and insert rows.
3. Run: `SELECT id FROM t WHERE length(read_blob(image_bytes)) = 11`.
4. Observe INTERNAL_ERROR.
**Environment:**
- Hudi version: 1.2.0-rc2
- Query engine: Spark 3.5
- Found during: 1.2.0-rc2
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the SQL example on the Spark 3.5 bundle, then trace how read_blob() is handled when it appears inside a WHERE predicate versus the SELECT list. Update the rejection so this unsupported use produces a clear analyzer error, and update the AI quick start with the .length workaround; verify that the query no longer fails with INTERNAL_ERROR.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- databases, documentation
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100