aws-samples / aws-samples/iceberg-streaming-examples
Check Remove duplicates functionality from SparkIcebergUtils
Open
- Dominant language
- Java
- Stars
- 31
- Forks
- 7
- PR merge metrics
- No merged PRs in 30d
Description
` spark
.sql("""
INSERT OVERWRITE employee
SELECT employee_id, start_date, first(team),first(role),first(address),first(name)
FROM employee
WHERE cast(start_date as date) = '2020-07-01' -- here we remove from a predefined day
GROUP BY employee_id, start_date
"""
)
.show();`
Contributor guide
Assessment
This issue has not been assessed yet.