apache / apache/beam

Can not insert into BigQuery table that is not empty

Open
#19,295 1 comment 0 reactions 0 assignees View on GitHub
bug dsl P3 sql
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

There is a Exception when I try to insert into BigQuery table that is not empty.

Example code Beam SQL:
```

BeamSqlRelUtils.toPCollection(pipeline, sqlEnv.parseQuery("INSERT INTO D_CARD_LITE(DIM_ID) VALUES('xxxx')"))
```

The exception messages as below:
```

java.lang.IllegalStateException: BigQuery table is not empty: mydataset:samples.D_CARD_LITE.
at com.google.common.base.Preconditions.checkState(Preconditions.java:518)

at org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers.verifyTableNotExistOrEmpty(BigQueryHelpers.java:470)

at org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO$Write.validate(BigQueryIO.java:1564)
at org.apache.beam.sdk.Pipeline$ValidateVisitor.enterCompositeTransform(Pipeline.java:641)

at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:645)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)

at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:311)

at org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:245)
at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458)

at org.apache.beam.sdk.Pipeline.validate(Pipeline.java:577)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:312)

at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
```

Imported from Jira [BEAM-6298](https://issues.apache.org/jira/browse/BEAM-6298). Original Jira may contain additional context.
Reported by: luat.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.