apache / apache/beam

SQL UDFs don't work well with Row as an argument

Open
#20,727 0 comments 0 reactions 0 assignees View on GitHub
dsl new feature P3 sql
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

SQL UDFs get a  [WrappedRow](https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamCalcRel.java#L642), which is suppose to be an internal wrapper for BeamCalcRel. We should revisit how rows are passed to UDFs, possibly passing the Schema Row instead. At very least users need to be able to get columns by name.

Imported from Jira [BEAM-11871](https://issues.apache.org/jira/browse/BEAM-11871). Original Jira may contain additional context.
Reported by: apilloud.

Contributor guide

Open the contributing guide

Research direction

Start in sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamCalcRel.java around the WrappedRow handling at line 642, and review the linked Jira issue for context. Determine how SQL UDF arguments expose row columns; done means users can access the relevant columns by name without relying on the internal wrapper.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.