apache / apache/iceberg

Core: Puffin LZ4 footer compression throws UnsupportedOperationException at runtime

Open
#16,033 5 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Apache Iceberg version

1.10.1 (latest release)

### Query engine

Other

### Please describe the bug 🐞

`PuffinFormat.FOOTER_COMPRESSION_CODEC` is hardcoded to `PuffinCompressionCodec.LZ4` (line 86), and `Puffin.WriteBuilder.compressFooter()` enables footer compression. However, both `PuffinFormat.compress()` and `PuffinFormat.decompress()` fall through to `throw new UnsupportedOperationException("Unsupported codec: lz4")` for the LZ4 case:

```java
case LZ4:
// TODO requires LZ4 frame compressor, e.g.
// https://github.com/airlift/aircompressor/pull/142
break;
```

This means:
1. Any user calling `Puffin.write(outputFile).compressFooter()` gets an `UnsupportedOperationException` at write time with no compile-time warning.
2. Reading any Puffin file written with LZ4 footer compression (e.g., from another implementation) will also fail.

The referenced aircompressor PR (https://github.com/airlift/aircompressor/pull/142) has been merged, so the dependency is available.

### Expected behavior

Either:
- Implement LZ4 frame compression/decompression using the now-available aircompressor library, or
- Throw a clear error in `Puffin.WriteBuilder.compressFooter()` indicating LZ4 is not yet supported, rather than failing deep in the write path

### Affected files

- PuffinFormat.java (lines 86, 111–114, 133–135)
- Puffin.java (line 69–70)

### Willingness to contribute

- [x] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time

Contributor guide

Open the contributing guide

Research direction

Start in PuffinFormat.java by reading the FOOTER_COMPRESSION_CODEC setting and the compress() and decompress() LZ4 branches, then inspect Puffin.java around WriteBuilder.compressFooter(). Verify both writing and reading behavior for LZ4 footer compression; done means supported round trips work with the available dependency, or the builder reports a clear unsupported-feature error.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.