apache / apache/iceberg

Temporarily reject statistics files for encrypted tables?

Open
#17,995 3 comments 0 reactions 0 assignees View on GitHub
improvement
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Feature Request / Improvement

Wary that 1.12 is coming up, and was auditing the state of encryption. I see https://github.com/apache/iceberg/pull/17533 and https://github.com/apache/iceberg/pull/17417 which are great. I'm not too opinionated here but I was wondering if we can temporarily throw on writing these files for encrypted tables - with those guards then simply being removed when the implementation lands?

I'm not opinionated here - just wondering if being defensive here improves / hardens the Spark encryption integration. I've put up https://github.com/apache/iceberg/pull/17994 and https://github.com/apache/iceberg/pull/17993, as the small changes here if folks are on board, but if not, happy to close these all out.

### Query engine

None

### Willingness to contribute

- [x] I can contribute this improvement/feature independently
- [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time

Contributor guide

Open the contributing guide

Research direction

Start by reviewing pull requests 17994 and 17993, then read the related encryption work in pull requests 17533 and 17417. Determine whether encrypted tables should reject writing statistics files temporarily, and confirm the expected behavior with the maintainers before proceeding.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark
Domain
data-engineering, security
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.