googleapis / googleapis/google-cloud-java

[java-bigquery] BigQuery: METASTORE format not supported.

オープン
#12,159 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る
api: bigquery priority: p3
主要言語
Java
スター
2.1k
フォーク
1.2k
平均マージ
1日 23時間
マージ済み PR(30日)
154

説明

Recently, while using the library, a new error have appeared
```
j.l.IllegalArgumentException: Format METASTORE is not supported
at c.g.c.b.TableDefinition.fromPb(TableDefinition.java:174)
```

That error happens when trying to access a bigquery table that doesn't belong to one of the [expected formats](https://github.com/googleapis/java-bigquery/blob/main/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/)

```
TableDefinition.java#L174)
default: throw new IllegalArgumentException("Format " + tablePb.getType() + " is not supported");
```

Is this something that it is expected to be supported soon?

#### Environment details

1. Specify the API at the beginning of the title: BigQuery
2. OS type and version: Irrelevant
3. Java version: All java versions
4. version(s): all versions

#### Steps to reproduce

1. Run the following code to get the list of bigquery tables and their metadata.

#### Code example

```scala
private val client: BigQuery = BigQueryOptions.newBuilder.setProjectId(projectId).build().getService

client.listDatasets(projectId).iterateAll().asScala.map(_.getDatasetId)
```

#### Stack trace
```
j.l.IllegalArgumentException: Format METASTORE is not supported
at c.g.c.b.TableDefinition.fromPb(TableDefinition.java:174)
at c.g.c.b.TableInfo$BuilderImpl.(TableInfo.java:253)
at c.g.c.bigquery.Table.fromPb(Table.java:690)
at c.g.c.b.BigQueryImpl$22.apply(BigQueryImpl.java:966)
at c.g.c.b.BigQueryImpl$22.apply(BigQueryImpl.java:963)
at c.g.c.c.Iterators$6.transform(Iterators.java:829)
at c.g.c.c.TransformedIterator.next(TransformedIterator.java:52)
at c.g.c.PageImpl$PageIterator.computeNext(PageImpl.java:72)
at c.g.c.c.AbstractIterator.tryToComputeNext(AbstractIterator.java:145)
at c.g.c.c.AbstractIterator.hasNext(AbstractIterator.java:140)
at s.c.c.JavaCollectionWrappers$JIteratorWrapper.hasNext(JavaCollectionWrappers.scala:46)
at s.c.StrictOptimizedIterableOps.map(StrictOptimizedIterableOps.scala:99)
at s.c.StrictOptimizedIterableOps.map$(StrictOptimizedIterableOps.scala:87)
at s.c.c.JavaCollectionWrappers$JIterableWrapper.map(JavaCollectionWrappers.scala:83)
```

#### External references such as API reference guides

This seems to be related to the new Iceberg formats
https://medium.com/@cesar.cordoba/iceberg-format-a-short-introduction-6cca39e48ac
https://cloud.google.com/bigquery/docs/iceberg-tables

#### Any additional information below

Checking the information from the command line over that dataset
```
bq ls --format=json --project_id="project_id" "dataset"
```

We can verify several tables have the "METASTORE" format.

```
{"kind":"bigquery#table","id":"project_id:dataset.mytable","tableReference":{"projectId":"project_id","datasetId":"dataset","tableId":"mytable"},"type":"METASTORE","creationTime":"1711215011000"},
```

Checking that table:
```
bq show --format=json "project_id:dataset.mytable"
```

```
"type":"METASTORE","externalDataConfiguration":{"sourceUris":["s3://edited.metadata.json"],"sourceFormat":"ICEBERG"
```

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。