Run WindowedWordCount Integration Test in Spark Streaming
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
The purpose of running WindowedWordCountIT in Spark is to have a streaming test pipeline running in Jenkins pre-commit using TestSparkRunner.
More discussion happened here:
https://github.com/apache/incubator-beam/pull/1045#issuecomment-251531770
Imported from Jira [BEAM-719](https://issues.apache.org/jira/browse/BEAM-719). Original Jira may contain additional context.
Reported by: markflyhigh.
This issue has child subcomponents which were not migrated over. See the original Jira for more information.
Contributor guide
Research direction
Start by locating WindowedWordCountIT and TestSparkRunner, then inspect how Jenkins pre-commit currently runs integration tests. Done means the WindowedWordCount streaming pipeline runs through TestSparkRunner as a Jenkins pre-commit test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- ci-cd, stream-processing, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100