[Feature Request]: BigQuery input needs support for RECORD data type
Open
Beam
P3
- Dominant language
- Java
- Stars
- 1.5k
- Forks
- 476
- Avg merge
- 19h
- Merged PRs (30d)
- 227
Description
### What would you like to happen?
When querying tabes in the Beam BigQuery Input transform, it's not possible to detect or query fields of type RECORD. It would be nice if Hop handled these nested fields with more grace. One possibility would be to give back the record fields as JSON strings.
### Issue Priority
Priority: 3
### Issue Component
Component: Beam
Contributor guide
Research direction
Start at the Beam BigQuery Input transform and inspect how BigQuery schemas and fields are detected and queried. Determine how RECORD fields should be represented, then add coverage showing that nested fields can be queried and handled consistently, with JSON strings as one proposed output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100