Expand Testing Beam pipeline documentation
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
User feedback:
"""
Feedback for documentation on Testing Beam pipeline
The documentation is not at all detailed and is pretty shallow, hardly covering most concepts to effectively design test-suit for an enterprise level pipeline.
For example, some the missing information are –
How to test the pipeline when the PTransform outputs tuple tags
How to design a negative test case where PTransform throws an exception
How to examine the data of the PCollection in PAssert. Something better than containsInAnyOrder is required.
Please note that the list is not exhaustive!
URL -
https://beam.apache.org/documentation/pipelines/test-your-pipeline/
"""
Imported from Jira [BEAM-12682](https://issues.apache.org/jira/browse/BEAM-12682). Original Jira may contain additional context.
Reported by: altay.
Contributor guide
Research direction
Start with the existing Testing Beam pipeline page at documentation/pipelines/test-your-pipeline/ and review how its current concepts are organized. Use the requested cases—tuple tags, exceptions from PTransforms, and inspecting PCollection data with PAssert—as the initial coverage checklist. Done means the page explains these testing scenarios in enough detail for enterprise-level test design, while accounting for the broader missing coverage noted in the issue.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation, testing
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100