apache / apache/hudi

e2e test module

Open
#16,765 1 comment 0 reactions 1 assignee Claimed by @ad1happy2go View on GitHub
from-jira priority:medium type:improvement
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

Currently, there is a lack of end-to-end tests combining Flink and Spark (Hive, etc.).

The most interesting scenario is stream writing by Flink and subsequent read by Spark. Also, we could schedule compaction for MOR table using Flink, and run compaction itself by Spark.

To implement it, I suppose, we need to add a separate module, with all needed Hudi, Flink, and Spark dependencies.

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-8788
- Type: Improvement
- Fix version(s):
- 1.1.0

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.