apache / apache/iceberg

`initial-default`+`write-default` `{}` (struct default value) representation issue

Open
#16,596 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Apache Iceberg version

1.11.0 (latest release)

### Query engine

None

### Please describe the bug 🐞

This issue is similar to https://github.com/apache/iceberg/issues/15932

[The spec](https://iceberg.apache.org/spec/#default-values) has a very clear table listing behavior for struct defaults, and makes it clear that a `struct` column can either have `null` as its default value, or `{}`, the default values for its fields should be decomposed.
> A non-null default is stored by setting `initial-default` or `write-default` to an empty struct (`{}`) that will use field values set from each field's `initial-default` or `write-default`, respectively.

But in the Iceberg REST API spec, both `initial-default` and `write-default` are modeled as a `PrimitiveTypeValue`:
```yaml
initial-default:
$ref: "#/components/schemas/PrimitiveTypeValue"
write-default:
$ref: "#/components/schemas/PrimitiveTypeValue"
```

So I don't think we can currently represent `{}` for an `initial-default` or `write-default`

Example request:
```json
{
"stage-create": true,
"name": "tbl",
"schema": {
"type": "struct",
"fields": [
{
"name": "col",
"id": 1,
"type": {
"type": "struct",
"fields": [
{
"name": "a",
"id": 2,
"type": "string",
"required": false,
"initial-default": "test"
},
{
"name": "b",
"id": 3,
"type": "int",
"required": false
}
]
},
"required": false,
"initial-default": {}
},
{
"name": "col2",
"id": 4,
"type": "int",
"required": false
}
],
"schema-id": 0
},
"partition-spec": {
"spec-id": 0,
"type": "struct",
"fields": []
},
"write-order": {
"order-id": 0,
"fields": []
},
"properties": {
"format-version": "3"
}
}
```

Attempting to POST this to the create table endpoint targeting an `iceberg-rest-fixture` catalog results in a long Internal Error message, of which an excerpt is:
```
Cannot create expression literal from org.apache.iceberg.data.GenericRecord: Record(null, null)
```

### Willingness to contribute

- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [x] I cannot contribute a fix for this bug at this time

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.