apache / apache/paimon

[Bug] interface for obtaining table information throw exception

Open
#5,110 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Paimon version

0.7.0-incubating
1.0.1

### Compute Engine

JavaAPI

### Minimal reproduce step

step1. create table using version 0.7.0-incubating

```
package com.hawk.paimon;

import org.apache.paimon.catalog.Catalog;
import org.apache.paimon.catalog.CatalogContext;
import org.apache.paimon.catalog.CatalogFactory;
import org.apache.paimon.catalog.Identifier;
import org.apache.paimon.fs.Path;
import org.apache.paimon.schema.Schema;
import org.apache.paimon.table.Table;
import org.apache.paimon.types.DataTypes;

import java.util.List;
import java.util.concurrent.TimeUnit;

public class CreateTable {

public static String tableName = "paimon_version_07";
public static void main(String[] args) {
CatalogContext catalogContext = CatalogContext.create(new Path("file:///tmp/paimon_version"));
try (Catalog catalog = CatalogFactory.createCatalog(catalogContext)){
Identifier identifier = Identifier.create("test",tableName);
List databases = catalog.listDatabases();
if (!databases.contains(identifier.getDatabaseName())){
catalog.createDatabase(identifier.getDatabaseName(), false);
}
List tables = catalog.listTables(identifier.getDatabaseName());
if (!tables.contains(identifier.getObjectName())){
Schema.Builder schemaBuilder = Schema.newBuilder();
schemaBuilder.column("id", DataTypes.SMALLINT());
schemaBuilder.column("name", DataTypes.STRING());
schemaBuilder.column("age", DataTypes.TINYINT());
// schemaBuilder.primaryKey("id");
schemaBuilder.comment("changelog table");
catalog.createTable(identifier, schemaBuilder.build(), false);
}
TimeUnit.SECONDS.sleep(1);
Table table = catalog.getTable(identifier);
System.out.println("===============================\n" + table.rowType());
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}

```

step2. run the above code again using version 1.0.1 throw exception

```
Caused by: java.lang.RuntimeException: You should define a 'bucket-key' for bucketed append mode.
at org.apache.paimon.schema.SchemaValidation.validateBucket(SchemaValidation.java:581)
at org.apache.paimon.schema.SchemaValidation.validateTableSchema(SchemaValidation.java:99)
at org.apache.paimon.table.AbstractFileStoreTable.copyInternal(AbstractFileStoreTable.java:351)
at org.apache.paimon.table.AbstractFileStoreTable.copy(AbstractFileStoreTable.java:295)
at org.apache.paimon.table.FileStoreTableFactory.createWithoutFallbackBranch(FileStoreTableFactory.java:127)
at org.apache.paimon.table.FileStoreTableFactory.create(FileStoreTableFactory.java:90)
at org.apache.paimon.table.FileStoreTableFactory.create(FileStoreTableFactory.java:80)
at org.apache.paimon.catalog.AbstractCatalog.getDataOrFormatTable(AbstractCatalog.java:423)
at org.apache.paimon.catalog.AbstractCatalog.getTable(AbstractCatalog.java:415)
at org.apache.paimon.catalog.CachingCatalog.getTable(CachingCatalog.java:254)
at com.hawk.paimon.CreateTable.main(CreateTable.java:37)
```

### What doesn't meet your expectations?

Primary key table is ok

APPendOnly table throw exception

version 0.8 and above is ok

### Anything else?
because the Schemaserializer #deserialize method performs the following logic when serializing version 0.7 of the appendOnly table schema
```
if (version <= 1 && !options.containsKey(CoreOptions.BUCKET.key())) {
options.put(CoreOptions.BUCKET.key(), "1");
}
```
Be changed to
```
if (schema.primaryKeys().isEmpty() && schema.bucketKeys().isEmpty() && !(schema.version() == TableSchema.PAIMON_07_VERSION && bucket == 1)) {
throw new RuntimeException(
"You should define a 'bucket-key' for bucketed append mode.");
}
```

_No response_

### Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at SchemaSerializer#deserialize and the SchemaValidation.validateBucket call shown in the stack trace, then reproduce the JavaAPI example with a table created in version 0.7.0-incubating and read by version 1.0.1. Done means the append-only table can be obtained without the bucket-key exception while primary-key tables continue to work.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.