googleapis / googleapis/google-cloud-java

[java-bigquery] BigQuery: METASTORE format not supported.

Đang mở
#12,159 1 bình luận 0 reaction 0 người được giao Xem trên GitHub
api: bigquery priority: p3
Ngôn ngữ chính
Java
Star
2.1k
Fork
1.2k
Merge trung bình
1 ngày 23 giờ
Pull request đã merge (30 ngày)
154

Mô tả

Recently, while using the library, a new error have appeared
```
j.l.IllegalArgumentException: Format METASTORE is not supported
at c.g.c.b.TableDefinition.fromPb(TableDefinition.java:174)
```

That error happens when trying to access a bigquery table that doesn't belong to one of the [expected formats](https://github.com/googleapis/java-bigquery/blob/main/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/)

```
TableDefinition.java#L174)
default: throw new IllegalArgumentException("Format " + tablePb.getType() + " is not supported");
```

Is this something that it is expected to be supported soon?

#### Environment details

1. Specify the API at the beginning of the title: BigQuery
2. OS type and version: Irrelevant
3. Java version: All java versions
4. version(s): all versions

#### Steps to reproduce

1. Run the following code to get the list of bigquery tables and their metadata.

#### Code example

```scala
private val client: BigQuery = BigQueryOptions.newBuilder.setProjectId(projectId).build().getService

client.listDatasets(projectId).iterateAll().asScala.map(_.getDatasetId)
```

#### Stack trace
```
j.l.IllegalArgumentException: Format METASTORE is not supported
at c.g.c.b.TableDefinition.fromPb(TableDefinition.java:174)
at c.g.c.b.TableInfo$BuilderImpl.(TableInfo.java:253)
at c.g.c.bigquery.Table.fromPb(Table.java:690)
at c.g.c.b.BigQueryImpl$22.apply(BigQueryImpl.java:966)
at c.g.c.b.BigQueryImpl$22.apply(BigQueryImpl.java:963)
at c.g.c.c.Iterators$6.transform(Iterators.java:829)
at c.g.c.c.TransformedIterator.next(TransformedIterator.java:52)
at c.g.c.PageImpl$PageIterator.computeNext(PageImpl.java:72)
at c.g.c.c.AbstractIterator.tryToComputeNext(AbstractIterator.java:145)
at c.g.c.c.AbstractIterator.hasNext(AbstractIterator.java:140)
at s.c.c.JavaCollectionWrappers$JIteratorWrapper.hasNext(JavaCollectionWrappers.scala:46)
at s.c.StrictOptimizedIterableOps.map(StrictOptimizedIterableOps.scala:99)
at s.c.StrictOptimizedIterableOps.map$(StrictOptimizedIterableOps.scala:87)
at s.c.c.JavaCollectionWrappers$JIterableWrapper.map(JavaCollectionWrappers.scala:83)
```

#### External references such as API reference guides

This seems to be related to the new Iceberg formats
https://medium.com/@cesar.cordoba/iceberg-format-a-short-introduction-6cca39e48ac
https://cloud.google.com/bigquery/docs/iceberg-tables

#### Any additional information below

Checking the information from the command line over that dataset
```
bq ls --format=json --project_id="project_id" "dataset"
```

We can verify several tables have the "METASTORE" format.

```
{"kind":"bigquery#table","id":"project_id:dataset.mytable","tableReference":{"projectId":"project_id","datasetId":"dataset","tableId":"mytable"},"type":"METASTORE","creationTime":"1711215011000"},
```

Checking that table:
```
bq show --format=json "project_id:dataset.mytable"
```

```
"type":"METASTORE","externalDataConfiguration":{"sourceUris":["s3://edited.metadata.json"],"sourceFormat":"ICEBERG"
```

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.