apache / apache/beam

Grouping By LogicalTypes is not supported

Open
#20,085 0 comments 0 reactions 0 assignees View on GitHub
bug core java P3 sql
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

Creating a schema from a BQ schema that has either TIME, DATE or DATETIME columns, and grouping by one of these fields throws NullPointerException.
```

Pipeline pipeline = Pipeline.create();

Schema beamSchemaWithLogicalTypes =
BigQueryUtils.fromTableSchema(

new TableSchema()
.setFields(
Arrays.asList(

new TableFieldSchema().setName("fTime").setType("TIME"),
new TableFieldSchema().setName("fDate").setType("DATE"),

new TableFieldSchema().setName("fDatetime").setType("DATETIME"))));

Row row =

Row.withSchema(beamSchemaWithLogicalTypes)
.addValues(
DateTime.parse("2020-02-02"),

DateTime.parse("2020-02-02"),
DateTime.parse("2020-02-02T00:00:00"))

.build();

PCollection outputRow =
pipeline
.apply(Create.of(row))
.setRowSchema(beamSchemaWithLogicalTypes)

.apply(
SqlTransform.query(
"SELECT fTime, fDate, fDatetime FROM
PCOLLECTION GROUP BY fTime, fDate, fDatetime"));

pipeline.run();

```

Imported from Jira [BEAM-9424](https://issues.apache.org/jira/browse/BEAM-9424). Original Jira may contain additional context.
Reported by: fdiazgon.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.