[Feature] Aggregation function to a nested struct type
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Motivation
The only aggregate function currently supported for nested types is nestd_update.
There is no support for STRUCT type yet. For spark example
```sql
CREATE TABLE t(
k INT,
a STRUCT,
g INT
)
TBLPROPERTIES ('primary-key' = 'k', 'merge-engine' = 'partial-update',
'fields.g.sequence-group'='a.a1,a.a2',
'fields.a.a1.aggregate-function'='sum',
'fields.a.a2.aggregate-function'='product'
)
INSERT INTO t
VALUES (1, STRUCT(1,3), 1);
INSERT INTO t
VALUES (1, STRUCT(2,4), 2);
```
The result expected is `1, STRUCT(3,12), 2`. However the actually output is `1, STRUCT(2,4), 2` because of nested aggregate-function has not been implemented.
### Solution
_No response_
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the existing nestd_update aggregate-function handling and the path that processes the Spark STRUCT example with fields.a.a1 and fields.a.a2 aggregation properties. Reproduce the two inserts from the issue and verify that the result becomes STRUCT(3,12) rather than STRUCT(2,4), while preserving the sequence and other fields.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- data-engineering, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100