apache / apache/beam

[Feature Request]: Support zstandard decompression on python runner

Open
#22,393 1 comment 0 reactions 1 assignee Claimed by @grufino View on GitHub
files io new feature P3 python
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What would you like to happen?

Currently it seems that CompressionTypes supported in the python runner are the following:
```
CompressionTypes.AUTO,
CompressionTypes.BZIP2,
CompressionTypes.DEFLATE,
CompressionTypes.GZIP,
CompressionTypes.UNCOMPRESSED
Unlike in the Java API, where the compression standard is supported.
```
I would like to add zstandard decompression both from a personal need and also as a way to start contributing.

Since this would be my first contribution maybe some guidance or references would be useful, I've been lurking around and seen that the testing may be a little complicated, not sure for this type of change. Or if someone else wants to take on this one also feel free.

### Issue Priority

Priority: 3

### Issue Component

Component: io-py-files

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.