[Feature Request]: Support for Objects and Array data types
- Dominant language
- Java
- Stars
- 1.5k
- Forks
- 476
- Avg merge
- 19h
- Merged PRs (30d)
- 227
Description
### What would you like to happen?
Migration: https://issues.apache.org/jira/browse/HOP-2476
As a developer I come across objects and arrays stored as part of a record fairly regularly. I'd like Hop to natively support them with dedicated data types so that extracting values becomes easier (no need to use any scripting or coding).
In example, similarly to Trifacta, Hop could offer easy to use functions like these ones then:
1) "Flatten array values into new rows":
[\{"type":"primary","colour":"green","product_number":3899},\{"type":"secondary","colour":"red","product_number":3899}]
returns two rows:
{"type":"primary","colour":"green","product_number":3899} {"type":"secondary","colour":"red","product_number":3899}
2) "Unnest values into new columns". The unnest transformation takes a nested key-value pair in the object and extracts it to a new column. e.g. unnest product_number:{"type":"secondary","colour":"red","product_number":3899} => 3899
### Issue Priority
Priority: 3
### Issue Component
Component: Metadata
Contributor guide
Research direction
No source files, tests, or entry points are named. Start by reviewing the Metadata component and migration HOP-2476, then clarify the design for native object and array types and their flattening and unnesting operations; done would require an agreed implementation scope and validation approach.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100