apache / apache/fluss

[Bug] Flink Catalog loses nested NOT NULL constraints in complex types

Open
#4,062 5 comments 0 reactions 1 assignee Claimed by @leekeiabstraction View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Fluss version

main (development)

### Please describe the bug 🐞

### Environment

- Flink 1.20.3
- Fluss Flink connector built from the current development branch

### Description

When a Fluss table is created through Flink SQL/Fluss Catalog, `NOT NULL` declared on fields nested inside `ROW`, `ARRAY>`, or similar complex types is lost.

For example:

```sql
CREATE TABLE nested_nullability_repro (
headers ARRAY>
) WITH (
'bucket.num' = '1'
);
```

Expected:

```text
headers ARRAY>
```

Actual schema retrieved from Fluss metadata:

```text
headers ARRAY>
```

Top-level `NOT NULL` constraints are retained; the problem affects nested fields.

### Minimal reproduction without Fluss

The same loss can be reproduced in Flink 1.20.3 with the default in-memory Catalog, before the schema reaches `FlinkCatalog`:

```java
TableEnvironment tableEnvironment =
TableEnvironment.create(EnvironmentSettings.newInstance().inBatchMode().build());

tableEnvironment.executeSql(
"CREATE TABLE nested_nullability_repro ("
+ "headers ARRAY>) "
+ "WITH ('connector' = 'blackhole')");

CatalogBaseTable table =
tableEnvironment
.getCatalog(tableEnvironment.getCurrentCatalog())
.get()
.getTable(
new ObjectPath(
tableEnvironment.getCurrentDatabase(),
"nested_nullability_repro"));

System.out.println(table.getUnresolvedSchema());
```

Output:

```text
(
`headers` ARRAY>
)
```

This indicates that nested nullability has already been lost in Flink's SQL/Catalog schema representation before the Fluss connector converts the type.

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.