apache / apache/beam

Make TextIOTest and AvroIOTest runner-agnostic

Open
#17,998 0 comments 0 reactions 0 assignees View on GitHub
bug core java P3
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

PipelineOptions contains a tempLocation property, and IOChannelUtils should be capable of handling arbitrary file locations that are used as a tempLocation.

The read and write tests for TextIO and AvroIO can use these properties to be written in a runner-agnostic fashion, and then be marked as RunnableOnService. Doing so allows all runners integrated with RunnableOnService to benefit from the existing tests.

Imported from Jira [BEAM-181](https://issues.apache.org/jira/browse/BEAM-181). Original Jira may contain additional context.
Reported by: tgroh.

Contributor guide

Open the contributing guide

Research direction

Start with TextIOTest and AvroIOTest, then read PipelineOptions and IOChannelUtils to understand how tempLocation and arbitrary file locations are handled. Update the read and write tests to use these properties without depending on a specific runner, mark them RunnableOnService, and verify that the shared tests run successfully across integrated runners.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.