googleapis / googleapis/google-cloud-java

[java-bigquery] BigQuery: METASTORE format not supported.

Abierto
#12,159 1 comentario 0 reacciones 0 asignados Ver en GitHub
api: bigquery priority: p3
Lenguaje dominante
Java
Estrellas
2.1k
Forks
1.2k
Merge medio
1 d 23 h
PR fusionados (30 d)
154

Descripción

Recently, while using the library, a new error have appeared
```
j.l.IllegalArgumentException: Format METASTORE is not supported
at c.g.c.b.TableDefinition.fromPb(TableDefinition.java:174)
```

That error happens when trying to access a bigquery table that doesn't belong to one of the [expected formats](https://github.com/googleapis/java-bigquery/blob/main/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/)

```
TableDefinition.java#L174)
default: throw new IllegalArgumentException("Format " + tablePb.getType() + " is not supported");
```

Is this something that it is expected to be supported soon?

#### Environment details

1. Specify the API at the beginning of the title: BigQuery
2. OS type and version: Irrelevant
3. Java version: All java versions
4. version(s): all versions

#### Steps to reproduce

1. Run the following code to get the list of bigquery tables and their metadata.

#### Code example

```scala
private val client: BigQuery = BigQueryOptions.newBuilder.setProjectId(projectId).build().getService

client.listDatasets(projectId).iterateAll().asScala.map(_.getDatasetId)
```

#### Stack trace
```
j.l.IllegalArgumentException: Format METASTORE is not supported
at c.g.c.b.TableDefinition.fromPb(TableDefinition.java:174)
at c.g.c.b.TableInfo$BuilderImpl.(TableInfo.java:253)
at c.g.c.bigquery.Table.fromPb(Table.java:690)
at c.g.c.b.BigQueryImpl$22.apply(BigQueryImpl.java:966)
at c.g.c.b.BigQueryImpl$22.apply(BigQueryImpl.java:963)
at c.g.c.c.Iterators$6.transform(Iterators.java:829)
at c.g.c.c.TransformedIterator.next(TransformedIterator.java:52)
at c.g.c.PageImpl$PageIterator.computeNext(PageImpl.java:72)
at c.g.c.c.AbstractIterator.tryToComputeNext(AbstractIterator.java:145)
at c.g.c.c.AbstractIterator.hasNext(AbstractIterator.java:140)
at s.c.c.JavaCollectionWrappers$JIteratorWrapper.hasNext(JavaCollectionWrappers.scala:46)
at s.c.StrictOptimizedIterableOps.map(StrictOptimizedIterableOps.scala:99)
at s.c.StrictOptimizedIterableOps.map$(StrictOptimizedIterableOps.scala:87)
at s.c.c.JavaCollectionWrappers$JIterableWrapper.map(JavaCollectionWrappers.scala:83)
```

#### External references such as API reference guides

This seems to be related to the new Iceberg formats
https://medium.com/@cesar.cordoba/iceberg-format-a-short-introduction-6cca39e48ac
https://cloud.google.com/bigquery/docs/iceberg-tables

#### Any additional information below

Checking the information from the command line over that dataset
```
bq ls --format=json --project_id="project_id" "dataset"
```

We can verify several tables have the "METASTORE" format.

```
{"kind":"bigquery#table","id":"project_id:dataset.mytable","tableReference":{"projectId":"project_id","datasetId":"dataset","tableId":"mytable"},"type":"METASTORE","creationTime":"1711215011000"},
```

Checking that table:
```
bq show --format=json "project_id:dataset.mytable"
```

```
"type":"METASTORE","externalDataConfiguration":{"sourceUris":["s3://edited.metadata.json"],"sourceFormat":"ICEBERG"
```

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.