apache / apache/beam

Provide a withReadableByteChannelFactory on TextIO.Read

Open
#18,578 0 comments 0 reactions 0 assignees View on GitHub
core improvement java P3
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
2d 2h
Merged PRs (30d)
205

Description

Hi,

I am prototyping a dataflow job that reads from GPG encrypted files. I got this working, but had to jump through some hoops to supply a new "compression" format to TextIO: I basically had to copy a good part of TextIO.Read only to add a `withDecompressingChannelFactory` method similar to `withWritableByteChannelFactory` in TextIO.Write.

Would it be possible to add a method `withDecompressingChannelFactory` or `withReadableByteChannelFactory` to TextIO.Read to allow users to plug in custom decompression/decryption code?

Imported from Jira [BEAM-3262](https://issues.apache.org/jira/browse/BEAM-3262). Original Jira may contain additional context.
Reported by: Tobi19.

Contributor guide

Open the contributing guide

Research direction

Start by reading the TextIO.Read API and compare it with the withWritableByteChannelFactory method on TextIO.Write. Determine how a readable channel factory should support custom decompression or decryption, and consider the issue complete when TextIO.Read exposes the requested factory hook for those use cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.