optional parameter is not supported in BeamSQL
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
I'm wring a UDF with optional parameters, the method code is:
```
public static String eval(String kvString, String keyName
, @Parameter(name = "pd", optional
= true) String pairSeperator
, @Parameter(name = "kvd", optional = true) String kvSeperator){
//...
}
```
And I see this error when assembling a Beam pipeline:
```
Execution Plan:
BeamProjectRel(ITEM_ID=[CAST(KV_EXT($0, 'itemId', DEFAULT(), DEFAULT())):BIGINT],
TRANSACTION_ID=[CAST(KV_EXT($0, 'transactionId', DEFAULT(), DEFAULT())):BIGINT], TRANSACTION_TIME=[KV_EXT($0,
'createdDT', DEFAULT(), DEFAULT())])
BeamIOSourceRel(table=[[....]])
Exception in thread "main"
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Operator: DEFAULT is not supported
yet!
at .........
Caused by: java.lang.UnsupportedOperationException: Operator: DEFAULT is not supported
yet!
at org.apache.beam.sdk.extensions.sql.impl.interpreter.BeamSqlFnExecutor.buildExpression(BeamSqlFnExecutor.java:425)
at
org.apache.beam.sdk.extensions.sql.impl.interpreter.BeamSqlFnExecutor.buildExpression(BeamSqlFnExecutor.java:202)
at
org.apache.beam.sdk.extensions.sql.impl.interpreter.BeamSqlFnExecutor.buildExpression(BeamSqlFnExecutor.java:202)
at
org.apache.beam.sdk.extensions.sql.impl.interpreter.BeamSqlFnExecutor.(BeamSqlFnExecutor.java:126)
at
org.apache.beam.sdk.extensions.sql.impl.rel.BeamProjectRel.buildBeamPipeline(BeamProjectRel.java:70)
at
org.apache.beam.sdk.extensions.sql.impl.planner.BeamQueryPlanner.compileBeamPipeline(BeamQueryPlanner.java:116)
at
org.apache.beam.sdk.extensions.sql.BeamSqlCli.compilePipeline(BeamSqlCli.java:112)
```
Imported from Jira [BEAM-3325](https://issues.apache.org/jira/browse/BEAM-3325). Original Jira may contain additional context.
Reported by: mingmxu.
Contributor guide
Research direction
Start at BeamSqlFnExecutor.java:425, where DEFAULT is rejected, and trace how the expression is reached from BeamProjectRel and BeamQueryPlanner. Determine how optional UDF parameters should be assembled, then verify that the shown BeamSQL pipeline compiles without the unsupported DEFAULT error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- backend, data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100