googleapis / googleapis/google-cloud-java
[java-bigquery] BigQuery: METASTORE format not supported.
- Langage dominant
- Java
- Étoiles
- 2.1k
- Forks
- 1.2k
- Merge moyen
- 1 j 23 h
- PR mergées (30 j)
- 154
Description
Recently, while using the library, a new error have appeared
```
j.l.IllegalArgumentException: Format METASTORE is not supported
at c.g.c.b.TableDefinition.fromPb(TableDefinition.java:174)
```
That error happens when trying to access a bigquery table that doesn't belong to one of the [expected formats](https://github.com/googleapis/java-bigquery/blob/main/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/)
```
TableDefinition.java#L174)
default: throw new IllegalArgumentException("Format " + tablePb.getType() + " is not supported");
```
Is this something that it is expected to be supported soon?
#### Environment details
1. Specify the API at the beginning of the title: BigQuery
2. OS type and version: Irrelevant
3. Java version: All java versions
4. version(s): all versions
#### Steps to reproduce
1. Run the following code to get the list of bigquery tables and their metadata.
#### Code example
```scala
private val client: BigQuery = BigQueryOptions.newBuilder.setProjectId(projectId).build().getService
client.listDatasets(projectId).iterateAll().asScala.map(_.getDatasetId)
```
#### Stack trace
```
j.l.IllegalArgumentException: Format METASTORE is not supported
at c.g.c.b.TableDefinition.fromPb(TableDefinition.java:174)
at c.g.c.b.TableInfo$BuilderImpl.(TableInfo.java:253)
at c.g.c.bigquery.Table.fromPb(Table.java:690)
at c.g.c.b.BigQueryImpl$22.apply(BigQueryImpl.java:966)
at c.g.c.b.BigQueryImpl$22.apply(BigQueryImpl.java:963)
at c.g.c.c.Iterators$6.transform(Iterators.java:829)
at c.g.c.c.TransformedIterator.next(TransformedIterator.java:52)
at c.g.c.PageImpl$PageIterator.computeNext(PageImpl.java:72)
at c.g.c.c.AbstractIterator.tryToComputeNext(AbstractIterator.java:145)
at c.g.c.c.AbstractIterator.hasNext(AbstractIterator.java:140)
at s.c.c.JavaCollectionWrappers$JIteratorWrapper.hasNext(JavaCollectionWrappers.scala:46)
at s.c.StrictOptimizedIterableOps.map(StrictOptimizedIterableOps.scala:99)
at s.c.StrictOptimizedIterableOps.map$(StrictOptimizedIterableOps.scala:87)
at s.c.c.JavaCollectionWrappers$JIterableWrapper.map(JavaCollectionWrappers.scala:83)
```
#### External references such as API reference guides
This seems to be related to the new Iceberg formats
https://medium.com/@cesar.cordoba/iceberg-format-a-short-introduction-6cca39e48ac
https://cloud.google.com/bigquery/docs/iceberg-tables
#### Any additional information below
Checking the information from the command line over that dataset
```
bq ls --format=json --project_id="project_id" "dataset"
```
We can verify several tables have the "METASTORE" format.
```
{"kind":"bigquery#table","id":"project_id:dataset.mytable","tableReference":{"projectId":"project_id","datasetId":"dataset","tableId":"mytable"},"type":"METASTORE","creationTime":"1711215011000"},
```
Checking that table:
```
bq show --format=json "project_id:dataset.mytable"
```
```
"type":"METASTORE","externalDataConfiguration":{"sourceUris":["s3://edited.metadata.json"],"sourceFormat":"ICEBERG"
```
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.