apache / apache/beam

bigqueryio for Go is doesn't support standard SQL

Open
#20,107 0 comments 0 reactions 0 assignees View on GitHub
gcp go improvement io P2
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

It appears that bigqueryio for Go is hardcoded to use legacy SQL, with no way to use standard SQL. The BQ API will still reject the request if you use the #stardardSQL comment/flag in the query since the sql version is set explicitly in the API call.

 

[https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/io/bigqueryio/bigquery.go#L127](https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/io/bigqueryio/bigquery.go#L127)

 

Seems like we can pass an options param into the Query function to provide this setting among others. I will take a look at doing this. LMK if there is another stylistic approach to consider.

Imported from Jira [BEAM-9243](https://issues.apache.org/jira/browse/BEAM-9243). Original Jira may contain additional context.
Reported by: sdotz.

Contributor guide

Open the contributing guide

Research direction

Start in sdks/go/pkg/beam/io/bigqueryio/bigquery.go around line 127, where the BigQuery query API call sets the SQL version. Trace the Query entry point and its request options, then verify that callers can select standard SQL while preserving the existing legacy SQL behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, google-cloud
Domain
cloud, data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.